-- Logs begin at Wed 2022-08-24 02:44:19 UTC, end at Wed 2022-08-24 04:14:30 UTC. -- Aug 24 03:28:54.272822 dv-cinder-854260-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 03:28:54.282404 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 03:28:54 2022] *** Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: nodename: dv-cinder-854260-1-iscsi Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: machine: x86_64 Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: clock source: unix Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: pcre jit disabled Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: detected number of CPU cores: 4 Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: current working directory: / Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: detected binary path: /usr/bin/uwsgi-core Aug 24 03:28:54.284427 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: your processes number limit is 47680 Aug 24 03:28:54.285155 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: your memory page size is 4096 bytes Aug 24 03:28:54.285155 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: detected max file descriptor number: 2048 Aug 24 03:28:54.285155 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: lock engine: pthread robust mutexes Aug 24 03:28:54.285155 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: thunder lock: enabled Aug 24 03:28:54.285155 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 03:28:54.285155 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 03:28:54.323857 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: Python main interpreter initialized at 0x55a0c00799c0 Aug 24 03:28:54.323857 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: python threads support enabled Aug 24 03:28:54.323857 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: your server socket listen backlog is limited to 100 connections Aug 24 03:28:54.323857 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: your mercy for graceful operations on workers is 90 seconds Aug 24 03:28:54.324226 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 03:28:54.324304 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: *** Operational MODE: preforking *** Aug 24 03:28:54.324550 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: *** uWSGI is running in multiple interpreter mode *** Aug 24 03:28:54.324550 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: spawned uWSGI master process (pid: 81931) Aug 24 03:28:54.324761 dv-cinder-854260-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 03:28:54.326586 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: spawned uWSGI worker 1 (pid: 81942, cores: 1) Aug 24 03:28:54.326586 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: spawned uWSGI worker 2 (pid: 81943, cores: 1) Aug 24 03:28:54.326586 dv-cinder-854260-1-iscsi devstack@keystone.service[81931]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 03:28:58.711855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:28:58.711855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: from cryptography.utils import int_from_bytes Aug 24 03:28:58.712631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:28:58.712631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: from cryptography.utils import int_from_bytes Aug 24 03:28:58.713043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:28:58.713043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: from cryptography.utils import int_from_bytes Aug 24 03:28:58.713147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:28:58.713147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: from cryptography.utils import int_from_bytes Aug 24 03:28:58.763555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 03:28:58.763753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 03:28:58.763836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 03:28:58.763949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] command line args: [] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 03:28:58.764068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 03:28:58.764236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] command line args: [] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 03:28:58.764331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 03:28:58.764405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ================================================================================ {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 03:28:58.764482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 03:28:58.764633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] admin_token = **** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.764718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ================================================================================ {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 03:28:58.764884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] config_dir = [] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.764970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] admin_token = **** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.765076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.765155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] config_dir = [] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.765262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] config_source = [] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.765364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.766050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] config_source = [] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.766177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] debug = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.766288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] debug = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.766375 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.766629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.766773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.766880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.766983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.767080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.767185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.767293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.767398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.767501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.767604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] list_limit = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.767716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] list_limit = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.767820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] log_config_append = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.767925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] log_config_append = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] log_dir = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] log_dir = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] log_file = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] log_file = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768666 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.768974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.769156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.769241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.769374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.769458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.769584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.769668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.769787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.769863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.769979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.770054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.770175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.770263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.770420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.770524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.770644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.770730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.770811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.770938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.771016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.771138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.771215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.771330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.771422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.771578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.771705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.82/identity {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.82/identity {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] publish_errors = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] publish_errors = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772581 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.772947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.773061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.773154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.773280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.773365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.773513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] standard_threads = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.773598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] standard_threads = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.773746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.773825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.773990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774183 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] use_journal = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] use_journal = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] use_json = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] use_json = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] use_stderr = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] use_stderr = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] use_syslog = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.774980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] use_syslog = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.775052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.775122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:28:58.775305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.775391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.775547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.775627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.775809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.775886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.776051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.776135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.776301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.776385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.776539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.776624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.776790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.776872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.777025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.777103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.777275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.777358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.777522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.777606 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.777773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.777852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.778009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.778087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.778257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.778343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.778493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.778590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.778760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.778844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.779009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.779105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.779245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.779324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.779503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.779586 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.779759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.779843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.780007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.780084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.780243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.780326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.780498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.780662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.780760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.780839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.781045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.781173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.781290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.781382 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.781527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.781690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.781768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.781935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.782012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.782171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.782300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.782411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.782583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.782678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.782842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.782930 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.783081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.783165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.783322 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.783406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.783575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.783661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.783827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.783905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.784063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.784167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.784314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.784427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.784548 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.784653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.784785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.784924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785237 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785777 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.785999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.786158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.786241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.786405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.786493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.786658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.786747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.786897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.786981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.787129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.787211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.787365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.787454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.787598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.787678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.787848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.787930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.788079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.788162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.788317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.788406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.788552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.788637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.788787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.788867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.789022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.789101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.789257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.789349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.789487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.789565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.789735 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.789817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.789971 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.790061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.790210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.790290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.790443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.790612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.790728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.790808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.790965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.791044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.791197 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.791276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.791433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.791511 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.791669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.791754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.791936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.792021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.792174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.792253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.792426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.792506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.792668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.792745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.792905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.792983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.793141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.793221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.793377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.793455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.793627 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.793706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.793864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.793955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.794102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.794180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.794344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.794422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.794603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.794697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.794841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.794923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.795077 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.795157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.795312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.795400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.795549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.795628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.795814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.795892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.796054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.796139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.796297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.796376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.796535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.796625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.796902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.797063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.797145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.797373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.797608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.797694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.797841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.797928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.798079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.798158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.798312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.798389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.798569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.798665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.798807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.798886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.799049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.799146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.799283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.799372 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.799526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.799605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.799764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.799842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.800015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.800101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.800252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.800330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.800514 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.800592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.800773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] auth.external = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.800910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.801014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.801100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.801246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] auth.external = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.801324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.801480 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.801588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.801765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] auth.password = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.801855 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.802005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] auth.token = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.802091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.802237 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] auth.password = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.802316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.802471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] auth.token = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.802629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.802840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.802928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.803079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.803165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.803328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.803417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.803563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.803644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.803802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.803881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.804043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] credential.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.804132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.804274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.804353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.804511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] credential.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.804591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.804760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.804848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.805002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.805081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.805252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.805330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.805490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.805569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.805733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.805812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.805970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.806050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.806221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.806300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.806474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.806594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.806788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.806867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.807024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.807109 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.807292 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.807373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.807539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.807619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.807800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.807882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.808040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] federation.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.808121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.808279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.808365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.808516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] federation.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.808596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.808752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.808832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.808991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.809070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.809237 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.809317 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.809476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.809736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.809814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.809977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.810065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.810227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.810305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.810466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.810564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.810733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.810828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.810974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.811053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.811214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.811293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.811463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.811551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.811701 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.811781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.811940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.812024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.812190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.812839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.812980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.813059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.813224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.813309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.813540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.813671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.813775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.813882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.814000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.814119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.814253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.814343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.814571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.814656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.814763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.814843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.814996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.815090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.815228 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.815308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.815475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.815556 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.815724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.815805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.815971 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.816072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.816226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.816305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.816476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.816558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.816752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.816858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.816990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.817071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.817223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.817303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.817457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.817561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.817696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.817778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.817929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.818010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.818166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.818245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.818405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.818483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.818667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.818758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.818919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.819000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.819152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.819238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.819401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.819489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.819649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.819736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.819886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.819966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.820141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.820222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.820388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.820469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.820620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.820700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.820858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.820940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.821089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.821254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.821335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.821506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.821594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.821754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.821921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.822001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.822165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.822341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.822430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.822611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.822696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.822853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.822933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.823113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.823205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.823350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.823429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.823600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.823678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.823848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.823935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.824087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.824167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.824324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.824404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.824558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.824645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.824792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.824872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.825034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.825114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.825290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.825377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.825523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.825602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.825772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.825853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.826011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.826097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.826248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.826328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.826484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.826576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.826736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.826828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.826972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.827057 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.827205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.827283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.827438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.827525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.827669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.827749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.828585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.828839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.828919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.829090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.829334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.829414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.829580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.829658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.829813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.829894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.830060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.830140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.830301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.830381 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.830563 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.830657 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.830802 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.830881 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.831037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.831118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.831282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.831364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.831531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.831611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.831769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.831849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.832024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.832106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.832255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.832335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.832500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.832579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.832738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.832819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.832974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.833054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.833209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.833289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.833457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.833538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.833699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.833779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.833931 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.834011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.834169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.834251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.834410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] resource.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.834490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] resource.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.834681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.834762 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.834933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.835013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.835168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.835246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.835420 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.835499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.835670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.835759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.835907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.835987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.836154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.836234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.836390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.836480 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.836628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.836708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.836864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] role.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.836943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] role.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.837110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] role.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.837191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] role.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.837342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.837429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.837591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.837677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.837839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.837921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.838087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.838167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.838333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.838413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.838604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.838687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.838851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.838930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.839098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.839179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.839346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.839428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.839582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.839661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.839827 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.839905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.840076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.840156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.840321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.840399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.840566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.840655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.840990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.841076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.841229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.841309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.841481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.841569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.841730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.841809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.841979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.842059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.842228 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.842316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.842463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.842557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.842740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.842821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.842993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.843083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.843231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.843311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.843488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.843636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.843726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.843812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.843938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.844046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.844190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.844312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.844448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.844577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.844694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.844831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.844931 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.845073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.845154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.845315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.845395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.845550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.845631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] token.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.845791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] token.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.845870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.846032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.846113 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.846284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.846364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.846539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.846623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.846806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.846888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.847043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.847126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.847287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.847367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.847530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.847612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.847774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.847853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.848010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.848079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.848249 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.848331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.848484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.848565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.848718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.848804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.848968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.849049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.849196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.849274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.849438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.849524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.849686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.849765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.849986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.connection = **** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.850074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.850232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.850313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.connection = **** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.850480 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.850615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.850743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.850825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.850985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.851060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.851217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.851302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.851449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.851527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.851674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.851754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.851909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.851988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.852145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.852225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.852378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.852456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.852620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.852697 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.852851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.852931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.853091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.853170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.853323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.853401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.853566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.853648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.853799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.853878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.854035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.854112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.854268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.854348 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.854427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 03:28:58.854518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:28:58.854694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 03:28:58.884845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.885037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.885120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.885195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.885274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.885410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.894325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.894603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.922934 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.923153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.923421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.923660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.991758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:58.992125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.047445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.047445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.047997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.048241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.211763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.211998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.212122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.212220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.212302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.212439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.212530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.212661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.212764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.212883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.212968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.213102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.237869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.238044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.238140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.238216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.238328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.238413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.238594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.238698 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.238823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.238907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.239121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.239222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.239315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.239393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.257515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.257978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.276278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.276457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.276543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.276619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.276728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.276813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.276956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.277050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.277180 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.277263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.277398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.277482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.277612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81943) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.277695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:28:59.295193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.295407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:28:59.295498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.295672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:28:59.295801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.295947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:28:59.296030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.296205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:28:59.296287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.296482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.296662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.297346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.297744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.297848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.298060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.298168 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.307624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.307829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.307921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.308111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.309673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.309753 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.309818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.309879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.315968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.316286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.318936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.319231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.319315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.319529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.319626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.319850 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.319947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.320112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.320195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.320380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.320473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.320650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.320731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.320925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.321007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.321206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.321344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.321634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.321714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.321914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.322004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.322191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.345436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.345645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.346950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.347174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.347275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.347462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.351978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.352205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.352289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.352490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.352573 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.352774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.359780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.360077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.360159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.361199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.364247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.364547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.364838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.364948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.365130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.365240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.365414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.365525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.365696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.365810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.365974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.366090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.366246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.366360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.366664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.366933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.383941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.384227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.384603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.384890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.388273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.388840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.391711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.392016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.392178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.392488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.396376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.396774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.399605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.399987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.411557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.411667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.411751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.411751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.411891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.411983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.412070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.412148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.412232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.412232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.412380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.412455 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.421143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.421446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.422239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.422743 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.422743 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.422863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.423008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.423282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.429634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.430356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.431328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.431615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.431909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.432189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.434303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.434639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.434926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.435023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.435225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.440204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.440307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.440379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.440379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.440379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.448281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.448442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.448564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.448734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.448847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.449025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.449132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.449315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.449418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.449608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.449688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.449901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.449983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.450196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.465758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.466229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.469412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.470690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.472374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.473037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.473819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.474497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.475397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.475679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.475977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.476077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.476294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.476595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.482894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.483415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.485234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.487298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.488225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.488518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.488711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.488822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.489012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.489098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.489307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.489604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.497103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.497399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.497671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.497756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.497981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.498265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.504875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81943) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.505162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.505904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.506011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:28:59.506080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.506144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.506218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.506284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.506347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.506646 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.506750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.506828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.506918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.507041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.507128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.507323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.507409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.507655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.507747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.507896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.507981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.508180 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.508272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.508443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81943) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.508534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.508715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.509006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.509283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:28:59.544182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a0c00799c0 pid: 81943 (default app) Aug 24 03:28:59.544869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a0c00799c0 pid: 81942 (default app) Aug 24 03:28:59.548502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dba40-accd-4da1-b657-0739f99bbac6 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:59.548785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dba40-accd-4da1-b657-0739f99bbac6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:59.549059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dba40-accd-4da1-b657-0739f99bbac6 None None] PATH_INFO: `/v3` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:59.553260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1/1] 10.222.0.82 () {50 vars in 791 bytes} [Wed Aug 24 03:28:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:07.413998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc635ad-885c-4142-a5d4-c1758442f6b8 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:07.414603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc635ad-885c-4142-a5d4-c1758442f6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:07.414742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc635ad-885c-4142-a5d4-c1758442f6b8 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:07.418921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1/2] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:07.425665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:07.426116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:07.426369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:07.499009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: INFO dbcounter [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] Registered counter for database keystone Aug 24 03:29:07.521356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG oslo_db.sqlalchemy.engines [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81943) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 03:29:07.527771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writer thread running {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 03:29:07.712287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG passlib.handlers.bcrypt [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81943) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 03:29:07.712629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG passlib.handlers.bcrypt [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81943) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 03:29:07.768132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:07.769757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:07.901194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 2/3] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 03:29:08.341737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc635ad-885c-4142-a5d4-c1758442f6b8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:08.410851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: INFO dbcounter [None req-0fc635ad-885c-4142-a5d4-c1758442f6b8 None None] Registered counter for database keystone Aug 24 03:29:08.430156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG oslo_db.sqlalchemy.engines [None req-0fc635ad-885c-4142-a5d4-c1758442f6b8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81942) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 03:29:08.436347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writer thread running {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 03:29:08.555010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc635ad-885c-4142-a5d4-c1758442f6b8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:08.559255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f631da-ca55-4038-ad52-db347d883380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:08.559877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f631da-ca55-4038-ad52-db347d883380 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:08.560156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f631da-ca55-4038-ad52-db347d883380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:08.560429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f631da-ca55-4038-ad52-db347d883380 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:08.569107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f631da-ca55-4038-ad52-db347d883380 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:08.621423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e1f631da-ca55-4038-ad52-db347d883380 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 03:29:08.621601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f631da-ca55-4038-ad52-db347d883380 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:08.632079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 2/4] 10.222.0.82 () {58 vars in 1324 bytes} [Wed Aug 24 03:29:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:29:08.638148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:08.653211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a2222-55f6-4049-99cb-0f6d5f4fbea2 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:08.657787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f667f26-8c09-4fbe-a4a7-b1f2ac585f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:08.658916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f667f26-8c09-4fbe-a4a7-b1f2ac585f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:08.659205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f667f26-8c09-4fbe-a4a7-b1f2ac585f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:08.659483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f667f26-8c09-4fbe-a4a7-b1f2ac585f57 admin admin] PATH_INFO: `/v3/projects/0dfb110c668f41649b36f4c725749a3a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:08.661056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f667f26-8c09-4fbe-a4a7-b1f2ac585f57 admin admin] RBAC: Authorizing `identity:get_project(project_id=0dfb110c668f41649b36f4c725749a3a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:08.715456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9f667f26-8c09-4fbe-a4a7-b1f2ac585f57 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 03:29:08.715816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f667f26-8c09-4fbe-a4a7-b1f2ac585f57 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:08.720475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 3/5] 10.222.0.82 () {56 vars in 1207 bytes} [Wed Aug 24 03:29:08 2022] GET /identity/v3/projects/0dfb110c668f41649b36f4c725749a3a => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:10.596317 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e00696-c500-4651-b492-120dd9043194 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:10.596885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e00696-c500-4651-b492-120dd9043194 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:10.596885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e00696-c500-4651-b492-120dd9043194 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:10.601108 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 3/6] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:10.607361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a32e5e-a185-40a5-92b2-8f4ed6d9fdae None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:10.607766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a32e5e-a185-40a5-92b2-8f4ed6d9fdae None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:10.607921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a32e5e-a185-40a5-92b2-8f4ed6d9fdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:10.628443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f4a32e5e-a185-40a5-92b2-8f4ed6d9fdae None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:10.629987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f4a32e5e-a185-40a5-92b2-8f4ed6d9fdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:10.637886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 4/7] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 03:29:10.908491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e00696-c500-4651-b492-120dd9043194 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:10.918984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e00696-c500-4651-b492-120dd9043194 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:10.924086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207d257-f6e0-4dfd-85d8-213dab2bbffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:10.924695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207d257-f6e0-4dfd-85d8-213dab2bbffc admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:10.925004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207d257-f6e0-4dfd-85d8-213dab2bbffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:10.925284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207d257-f6e0-4dfd-85d8-213dab2bbffc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:10.933175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207d257-f6e0-4dfd-85d8-213dab2bbffc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:10.937185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207d257-f6e0-4dfd-85d8-213dab2bbffc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:10.944522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 4/8] 10.222.0.82 () {58 vars in 1324 bytes} [Wed Aug 24 03:29:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:29:10.950167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a32e5e-a185-40a5-92b2-8f4ed6d9fdae None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:10.959457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a32e5e-a185-40a5-92b2-8f4ed6d9fdae None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:10.963842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfe19f-a416-4f67-b8b9-6cf61c2ac390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:10.964469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfe19f-a416-4f67-b8b9-6cf61c2ac390 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:10.964754 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfe19f-a416-4f67-b8b9-6cf61c2ac390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:10.965031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfe19f-a416-4f67-b8b9-6cf61c2ac390 admin admin] PATH_INFO: `/v3/users/c44d1cb246ea41a89355002d9d2481af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:10.966333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfe19f-a416-4f67-b8b9-6cf61c2ac390 admin admin] RBAC: Authorizing `identity:get_user(user_id=c44d1cb246ea41a89355002d9d2481af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:10.968204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfe19f-a416-4f67-b8b9-6cf61c2ac390 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:10.969888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 5/9] 10.222.0.82 () {56 vars in 1198 bytes} [Wed Aug 24 03:29:10 2022] GET /identity/v3/users/c44d1cb246ea41a89355002d9d2481af => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:12.955021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78c5c5-4cfd-4175-9fa8-46d42712df12 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:12.955569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78c5c5-4cfd-4175-9fa8-46d42712df12 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:12.955569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78c5c5-4cfd-4175-9fa8-46d42712df12 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:12.956916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 5/10] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:12.962355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b84a4-e203-48a6-8a06-a4828e5884e8 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:12.962820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b84a4-e203-48a6-8a06-a4828e5884e8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:12.962956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b84a4-e203-48a6-8a06-a4828e5884e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:12.981561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-559b84a4-e203-48a6-8a06-a4828e5884e8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:12.982789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-559b84a4-e203-48a6-8a06-a4828e5884e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:12.995387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdba0e-d6ab-47a4-bdb5-4312d0ca7757 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:12.995825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdba0e-d6ab-47a4-bdb5-4312d0ca7757 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:12.996106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdba0e-d6ab-47a4-bdb5-4312d0ca7757 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:12.997432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 6/11] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:13.003011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649e9a8-34c5-416c-a683-1e5814d1b0e8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:13.003300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649e9a8-34c5-416c-a683-1e5814d1b0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:13.003574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649e9a8-34c5-416c-a683-1e5814d1b0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:13.052920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG passlib.handlers.bcrypt [None req-b649e9a8-34c5-416c-a683-1e5814d1b0e8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81942) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 03:29:13.053358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG passlib.handlers.bcrypt [None req-b649e9a8-34c5-416c-a683-1e5814d1b0e8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81942) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 03:29:13.074907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 6/12] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:13.100123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b649e9a8-34c5-416c-a683-1e5814d1b0e8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:13.101748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b649e9a8-34c5-416c-a683-1e5814d1b0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:13.207370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 7/13] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:29:13.342674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b84a4-e203-48a6-8a06-a4828e5884e8 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:13.368275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:13.449714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b84a4-e203-48a6-8a06-a4828e5884e8 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:13.480846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649e9a8-34c5-416c-a683-1e5814d1b0e8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:13.485661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5aa67-a31c-46d6-8eff-58319400605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:13.486329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5aa67-a31c-46d6-8eff-58319400605e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:13.486655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5aa67-a31c-46d6-8eff-58319400605e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:13.486942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5aa67-a31c-46d6-8eff-58319400605e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:13.499162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5aa67-a31c-46d6-8eff-58319400605e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-9bb5aa67-a31c-46d6-8eff-58319400605e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:13.508933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:13.509888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:13.511036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 7/14] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:29:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:13.517047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5aa67-a31c-46d6-8eff-58319400605e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:13.590560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649e9a8-34c5-416c-a683-1e5814d1b0e8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:13.592854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:13.601115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:13.619117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5aa67-a31c-46d6-8eff-58319400605e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:13.628738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e4426-e662-4eb9-a9d2-cc99da7f134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:13.629610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e4426-e662-4eb9-a9d2-cc99da7f134b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:13.629899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e4426-e662-4eb9-a9d2-cc99da7f134b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:13.630176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e4426-e662-4eb9-a9d2-cc99da7f134b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:13.654840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc628f-a4f5-4c26-b497-211fd9fe3552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:13.655452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc628f-a4f5-4c26-b497-211fd9fe3552 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:13.655739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc628f-a4f5-4c26-b497-211fd9fe3552 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:13.656168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc628f-a4f5-4c26-b497-211fd9fe3552 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:13.656903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc628f-a4f5-4c26-b497-211fd9fe3552 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:13.657211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc628f-a4f5-4c26-b497-211fd9fe3552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:13.658773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc628f-a4f5-4c26-b497-211fd9fe3552 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:13.670882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 8/15] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:13.676688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc628f-a4f5-4c26-b497-211fd9fe3552 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:13.704291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e4426-e662-4eb9-a9d2-cc99da7f134b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:13.705870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e4426-e662-4eb9-a9d2-cc99da7f134b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:13.779849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc628f-a4f5-4c26-b497-211fd9fe3552 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:13.808372 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 8/16] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:29:13.814204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e4426-e662-4eb9-a9d2-cc99da7f134b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:13.817047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:13.818276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2f754-3aa7-4b23-a52a-ee2b28ed65ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:13.818951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2f754-3aa7-4b23-a52a-ee2b28ed65ba None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:13.819240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2f754-3aa7-4b23-a52a-ee2b28ed65ba None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:13.819518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2f754-3aa7-4b23-a52a-ee2b28ed65ba None admin] PATH_INFO: `/v3/users/c44d1cb246ea41a89355002d9d2481af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:13.820695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2f754-3aa7-4b23-a52a-ee2b28ed65ba None admin] RBAC: Authorizing `identity:get_user(user_id=c44d1cb246ea41a89355002d9d2481af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:13.822200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2f754-3aa7-4b23-a52a-ee2b28ed65ba None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:13.823986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 9/17] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:13 2022] GET /identity/v3/users/c44d1cb246ea41a89355002d9d2481af => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:13.829898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2f754-3aa7-4b23-a52a-ee2b28ed65ba None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:13.845943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:13.913766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e4426-e662-4eb9-a9d2-cc99da7f134b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:13.939582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2f754-3aa7-4b23-a52a-ee2b28ed65ba None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:13.947015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69f6e-e7fd-47e5-869f-06425ef071e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:13.947874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69f6e-e7fd-47e5-869f-06425ef071e4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:13.948155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69f6e-e7fd-47e5-869f-06425ef071e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:13.948431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69f6e-e7fd-47e5-869f-06425ef071e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:13.967153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d69f6e-e7fd-47e5-869f-06425ef071e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:13.969290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d69f6e-e7fd-47e5-869f-06425ef071e4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:13.974342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41f2b2-8db3-4c06-b2a3-fe965a9d89f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:13.975005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41f2b2-8db3-4c06-b2a3-fe965a9d89f2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:13.975286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41f2b2-8db3-4c06-b2a3-fe965a9d89f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:13.975562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41f2b2-8db3-4c06-b2a3-fe965a9d89f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:13.976677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41f2b2-8db3-4c06-b2a3-fe965a9d89f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:13.978130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41f2b2-8db3-4c06-b2a3-fe965a9d89f2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:13.979715 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 10/18] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:13.986071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41f2b2-8db3-4c06-b2a3-fe965a9d89f2 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-32d69f6e-e7fd-47e5-869f-06425ef071e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:29:13.987518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:13.988379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:29:13.989335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:29:13.989335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:29:13.989335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:13.989335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 9/19] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:29:13.994137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69f6e-e7fd-47e5-869f-06425ef071e4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:14.053486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:14.076028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:14.095496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41f2b2-8db3-4c06-b2a3-fe965a9d89f2 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:14.096997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69f6e-e7fd-47e5-869f-06425ef071e4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:14.132534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec859450-125f-4224-89e1-811e8717c659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:14.132746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc709231-5f3b-45d5-abac-b284264c10a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:14.133153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859450-125f-4224-89e1-811e8717c659 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:14.133296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc709231-5f3b-45d5-abac-b284264c10a2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:14.133517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859450-125f-4224-89e1-811e8717c659 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:14.133614 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc709231-5f3b-45d5-abac-b284264c10a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:14.133896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859450-125f-4224-89e1-811e8717c659 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:14.134003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc709231-5f3b-45d5-abac-b284264c10a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:14.134531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec859450-125f-4224-89e1-811e8717c659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:14.134686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc709231-5f3b-45d5-abac-b284264c10a2 None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.domain.id=default, user.id=c44d1cb246ea41a89355002d9d2481af) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:14.134836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec859450-125f-4224-89e1-811e8717c659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:14.134980 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc709231-5f3b-45d5-abac-b284264c10a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:14.136256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec859450-125f-4224-89e1-811e8717c659 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:14.136419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc709231-5f3b-45d5-abac-b284264c10a2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:14.145150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 11/20] 10.222.0.82 () {56 vars in 1322 bytes} [Wed Aug 24 03:29:13 2022] GET /identity/v3/role_assignments?user.id=c44d1cb246ea41a89355002d9d2481af&scope.domain.id=default&role.id=9e904216086e4a06abf9216b0c863f89 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:14.160569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 10/21] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:16.285576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d3f19-a740-43bf-a062-0bc93030f486 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:16.286455 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d3f19-a740-43bf-a062-0bc93030f486 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:16.287006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d3f19-a740-43bf-a062-0bc93030f486 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:16.288577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 12/22] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:16.293929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a1484-575a-492b-9f82-a1238c9f407f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:16.294803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a1484-575a-492b-9f82-a1238c9f407f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:16.295295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a1484-575a-492b-9f82-a1238c9f407f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:16.320133 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-0c2a1484-575a-492b-9f82-a1238c9f407f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:16.321706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0c2a1484-575a-492b-9f82-a1238c9f407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:16.424963 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 11/23] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:29:16.606840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74674d-651b-4cce-a22e-fb1a6fb86698 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:16.607478 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74674d-651b-4cce-a22e-fb1a6fb86698 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:16.607977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74674d-651b-4cce-a22e-fb1a6fb86698 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:16.609606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 13/24] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:16.624131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9228d9-2748-40a4-a087-eea5890891aa None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:16.625278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9228d9-2748-40a4-a087-eea5890891aa None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:16.625780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9228d9-2748-40a4-a087-eea5890891aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:16.651615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-fe9228d9-2748-40a4-a087-eea5890891aa None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:16.653164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fe9228d9-2748-40a4-a087-eea5890891aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:16.713660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:16.745630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 12/25] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:29:16.860633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74674d-651b-4cce-a22e-fb1a6fb86698 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:16.995796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74674d-651b-4cce-a22e-fb1a6fb86698 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:17.047773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6c5a0-e907-4894-9a85-fd2a81ab1ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:17.048422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6c5a0-e907-4894-9a85-fd2a81ab1ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.048709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6c5a0-e907-4894-9a85-fd2a81ab1ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.048987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6c5a0-e907-4894-9a85-fd2a81ab1ac7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.070711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6c5a0-e907-4894-9a85-fd2a81ab1ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:17.074112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a0fe0-f2da-4bf7-a244-88f605601b73 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.074405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a0fe0-f2da-4bf7-a244-88f605601b73 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.074763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a0fe0-f2da-4bf7-a244-88f605601b73 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.076267 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 13/26] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-70c6c5a0-e907-4894-9a85-fd2a81ab1ac7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:17.080240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:17.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:17.081899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 14/27] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:29:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:17.088000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a68fa-ab64-44ab-8637-082aeed04c9b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.088296 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a68fa-ab64-44ab-8637-082aeed04c9b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.088580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a68fa-ab64-44ab-8637-082aeed04c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.096294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6c5a0-e907-4894-9a85-fd2a81ab1ac7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:17.103812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:17.113246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a43a68fa-ab64-44ab-8637-082aeed04c9b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:17.114388 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a43a68fa-ab64-44ab-8637-082aeed04c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:17.241913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6c5a0-e907-4894-9a85-fd2a81ab1ac7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:17.271856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 14/28] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:17.273131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52683ca2-db81-4b8f-a548-b89c333d2136 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.273580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52683ca2-db81-4b8f-a548-b89c333d2136 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.274000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52683ca2-db81-4b8f-a548-b89c333d2136 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.275746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 15/29] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:17.276611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52683ca2-db81-4b8f-a548-b89c333d2136 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:17.291141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67452b45-93f1-4ea4-bc7b-248e3881bee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:17.291761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67452b45-93f1-4ea4-bc7b-248e3881bee2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.292056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67452b45-93f1-4ea4-bc7b-248e3881bee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.292337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67452b45-93f1-4ea4-bc7b-248e3881bee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.293023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67452b45-93f1-4ea4-bc7b-248e3881bee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:17.293337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67452b45-93f1-4ea4-bc7b-248e3881bee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:17.294944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67452b45-93f1-4ea4-bc7b-248e3881bee2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:17.309083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 15/30] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:17.318455 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84937e23-f032-401e-8f27-d52ac482bfea None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.318990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84937e23-f032-401e-8f27-d52ac482bfea None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.319417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84937e23-f032-401e-8f27-d52ac482bfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.369957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-84937e23-f032-401e-8f27-d52ac482bfea None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:17.373007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-84937e23-f032-401e-8f27-d52ac482bfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:17.526399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:17.540479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 16/31] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:17.541390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84937e23-f032-401e-8f27-d52ac482bfea None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:17.545612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52683ca2-db81-4b8f-a548-b89c333d2136 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:17.570891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:17.592115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b174ca-6fec-4f52-ba91-f23befe29180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:17.592769 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b174ca-6fec-4f52-ba91-f23befe29180 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.593051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b174ca-6fec-4f52-ba91-f23befe29180 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.593326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b174ca-6fec-4f52-ba91-f23befe29180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.607518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b174ca-6fec-4f52-ba91-f23befe29180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:17.609037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b174ca-6fec-4f52-ba91-f23befe29180 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-34b174ca-6fec-4f52-ba91-f23befe29180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:29:17.641643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:17.642582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:29:17.643411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:29:17.643411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:29:17.643411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:17.648158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 16/32] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:17.649139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b174ca-6fec-4f52-ba91-f23befe29180 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:17.865940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84937e23-f032-401e-8f27-d52ac482bfea None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:17.883138 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b174ca-6fec-4f52-ba91-f23befe29180 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:17.934690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b4545-7c56-478f-a32c-edb42819f1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:17.935582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b4545-7c56-478f-a32c-edb42819f1ce None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.936008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b4545-7c56-478f-a32c-edb42819f1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.936441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b4545-7c56-478f-a32c-edb42819f1ce None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.938251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b4545-7c56-478f-a32c-edb42819f1ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:17.940742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b4545-7c56-478f-a32c-edb42819f1ce None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:17.945263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97537c36-2ac8-4acd-9770-59d925ea780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:17.946181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97537c36-2ac8-4acd-9770-59d925ea780d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.946725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97537c36-2ac8-4acd-9770-59d925ea780d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.947203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97537c36-2ac8-4acd-9770-59d925ea780d None admin] PATH_INFO: `/v3/users/c44d1cb246ea41a89355002d9d2481af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.948776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97537c36-2ac8-4acd-9770-59d925ea780d None admin] RBAC: Authorizing `identity:get_user(user_id=c44d1cb246ea41a89355002d9d2481af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:17.951059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97537c36-2ac8-4acd-9770-59d925ea780d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:17.953175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 17/33] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:17 2022] GET /identity/v3/users/c44d1cb246ea41a89355002d9d2481af => generated 247 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:17.954331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97537c36-2ac8-4acd-9770-59d925ea780d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:18.034842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:18.082167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97537c36-2ac8-4acd-9770-59d925ea780d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:18.119118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6acf1e-335e-441f-8041-018bb2af4cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:18.119735 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6acf1e-335e-441f-8041-018bb2af4cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:18.120022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6acf1e-335e-441f-8041-018bb2af4cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:18.120300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6acf1e-335e-441f-8041-018bb2af4cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:18.120947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6acf1e-335e-441f-8041-018bb2af4cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:18.121255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6acf1e-335e-441f-8041-018bb2af4cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:18.122667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6acf1e-335e-441f-8041-018bb2af4cf6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:18.140148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 18/34] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:18.141041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6acf1e-335e-441f-8041-018bb2af4cf6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:18.177548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 17/35] 10.222.0.82 () {60 vars in 1131 bytes} [Wed Aug 24 03:29:17 2022] POST /identity/v3/roles => generated 219 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:18.179268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b4545-7c56-478f-a32c-edb42819f1ce None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:18.204315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:18.259993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6acf1e-335e-441f-8041-018bb2af4cf6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:18.287497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:18.301502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b4545-7c56-478f-a32c-edb42819f1ce None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:18.302726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5991-3440-4c3c-871d-f932d67819d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:18.303987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5991-3440-4c3c-871d-f932d67819d4 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:18.303987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5991-3440-4c3c-871d-f932d67819d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:18.304322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5991-3440-4c3c-871d-f932d67819d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:18.305030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f5991-3440-4c3c-871d-f932d67819d4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:18.306618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f5991-3440-4c3c-871d-f932d67819d4 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:18.339824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e8186-ed00-4f06-8dc6-1c5b760ea67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:18.340474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e8186-ed00-4f06-8dc6-1c5b760ea67d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:18.340754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e8186-ed00-4f06-8dc6-1c5b760ea67d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:18.341072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e8186-ed00-4f06-8dc6-1c5b760ea67d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:18.342497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e8186-ed00-4f06-8dc6-1c5b760ea67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:18.346675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e8186-ed00-4f06-8dc6-1c5b760ea67d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:18.350997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 18/36] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:18.350997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0ef26-1a24-4e84-8efa-8082e321e22e None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:18.350997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0ef26-1a24-4e84-8efa-8082e321e22e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:18.350997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0ef26-1a24-4e84-8efa-8082e321e22e None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:18.350997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 19/37] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:18.351503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcec65e-99ba-4d92-9194-7fe9c977e512 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:18.351787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcec65e-99ba-4d92-9194-7fe9c977e512 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:18.352065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcec65e-99ba-4d92-9194-7fe9c977e512 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:18.353540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 20/38] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:18.354247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcec65e-99ba-4d92-9194-7fe9c977e512 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:18.448004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:18.464772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcec65e-99ba-4d92-9194-7fe9c977e512 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:18.499653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ade1e-629c-4c3f-9fc7-581daf773c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:18.500479 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ade1e-629c-4c3f-9fc7-581daf773c28 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:18.500763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ade1e-629c-4c3f-9fc7-581daf773c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:18.501040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ade1e-629c-4c3f-9fc7-581daf773c28 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:18.501688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ade1e-629c-4c3f-9fc7-581daf773c28 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:18.503196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ade1e-629c-4c3f-9fc7-581daf773c28 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:18.531830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 19/39] 10.222.0.82 () {60 vars in 1131 bytes} [Wed Aug 24 03:29:18 2022] POST /identity/v3/roles => generated 225 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:18.532790 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5991-3440-4c3c-871d-f932d67819d4 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:18.645918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5991-3440-4c3c-871d-f932d67819d4 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:18.681159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 21/40] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:29:18 2022] POST /identity/v3/projects => generated 301 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:18.682561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93513ac0-565e-424d-b195-724bab368324 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:18.682858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93513ac0-565e-424d-b195-724bab368324 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:18.683167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93513ac0-565e-424d-b195-724bab368324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:18.684561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73a3c5-6fd3-480e-852e-94bb14535281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:18.685510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73a3c5-6fd3-480e-852e-94bb14535281 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:18.686022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73a3c5-6fd3-480e-852e-94bb14535281 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:18.686527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73a3c5-6fd3-480e-852e-94bb14535281 None admin] PATH_INFO: `/v3/domains/default/users/c44d1cb246ea41a89355002d9d2481af/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:18.690036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73a3c5-6fd3-480e-852e-94bb14535281 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c44d1cb246ea41a89355002d9d2481af, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:18.692140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73a3c5-6fd3-480e-852e-94bb14535281 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:18.708020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-93513ac0-565e-424d-b195-724bab368324 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:18.710143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-93513ac0-565e-424d-b195-724bab368324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:18.791798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 20/41] 10.222.0.82 () {58 vars in 1361 bytes} [Wed Aug 24 03:29:18 2022] PUT /identity/v3/domains/default/users/c44d1cb246ea41a89355002d9d2481af/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:29:18.793467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226ef77-835f-4cf6-960b-f8938466212a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:18.793765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226ef77-835f-4cf6-960b-f8938466212a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:18.794042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226ef77-835f-4cf6-960b-f8938466212a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:18.800849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 22/42] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:18.937776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c226ef77-835f-4cf6-960b-f8938466212a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:18.939714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c226ef77-835f-4cf6-960b-f8938466212a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:18.987878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:19.017102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 21/43] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:19.081189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93513ac0-565e-424d-b195-724bab368324 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:19.150928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:19.185059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93513ac0-565e-424d-b195-724bab368324 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:19.221176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e3f40-7b30-4ff8-bf22-bf68220252df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:19.221798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e3f40-7b30-4ff8-bf22-bf68220252df None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:19.222089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e3f40-7b30-4ff8-bf22-bf68220252df None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:19.222362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e3f40-7b30-4ff8-bf22-bf68220252df None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:19.223062 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e3f40-7b30-4ff8-bf22-bf68220252df None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:19.224693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e3f40-7b30-4ff8-bf22-bf68220252df None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:19.288721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226ef77-835f-4cf6-960b-f8938466212a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:19.298793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 23/44] 10.222.0.82 () {60 vars in 1131 bytes} [Wed Aug 24 03:29:19 2022] POST /identity/v3/roles => generated 223 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:19.413136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226ef77-835f-4cf6-960b-f8938466212a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:19.460967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be697685-a094-47a2-ab24-f6b78aa5b4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:19.461617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be697685-a094-47a2-ab24-f6b78aa5b4ed None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:19.461901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be697685-a094-47a2-ab24-f6b78aa5b4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:19.462180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be697685-a094-47a2-ab24-f6b78aa5b4ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:19.463370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be697685-a094-47a2-ab24-f6b78aa5b4ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:19.464846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be697685-a094-47a2-ab24-f6b78aa5b4ed None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:19.466903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 22/45] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:19.473020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e3f40-7b30-4ff8-bf22-bf68220252df None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:19.578689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e3f40-7b30-4ff8-bf22-bf68220252df None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:19.604032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:19.612053 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78de3d-5cd6-49ec-8324-e396eb7a14fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:19.612684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78de3d-5cd6-49ec-8324-e396eb7a14fc None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:19.612974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78de3d-5cd6-49ec-8324-e396eb7a14fc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:19.613256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78de3d-5cd6-49ec-8324-e396eb7a14fc None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:19.613916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78de3d-5cd6-49ec-8324-e396eb7a14fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:19.615423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78de3d-5cd6-49ec-8324-e396eb7a14fc None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:19.759766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 24/46] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:29:19 2022] POST /identity/v3/projects => generated 312 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:20.780137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfdeb-78d8-4841-ac45-fe0a48d60594 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:20.780652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfdeb-78d8-4841-ac45-fe0a48d60594 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:20.780736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfdeb-78d8-4841-ac45-fe0a48d60594 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:20.782804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 23/47] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:20.788975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc1540-df41-42fa-9073-c67e27478d06 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:20.789417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc1540-df41-42fa-9073-c67e27478d06 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:20.789547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc1540-df41-42fa-9073-c67e27478d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:20.807586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-bafc1540-df41-42fa-9073-c67e27478d06 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:20.808785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bafc1540-df41-42fa-9073-c67e27478d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:20.881554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 25/48] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:21.166793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfdeb-78d8-4841-ac45-fe0a48d60594 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:21.221358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:21.279895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfdeb-78d8-4841-ac45-fe0a48d60594 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:21.314435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213899f-502b-4150-9f92-5a419e9691fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:21.315098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213899f-502b-4150-9f92-5a419e9691fa None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.315382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213899f-502b-4150-9f92-5a419e9691fa None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.315656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213899f-502b-4150-9f92-5a419e9691fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.326034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213899f-502b-4150-9f92-5a419e9691fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-0213899f-502b-4150-9f92-5a419e9691fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:21.330968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:21.331824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:21.332598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 24/49] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:21.339346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc1540-df41-42fa-9073-c67e27478d06 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:21.440503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc1540-df41-42fa-9073-c67e27478d06 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:21.451011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:21.475102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38924942-2a39-4029-b9c5-1e0662a5e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:21.475725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924942-2a39-4029-b9c5-1e0662a5e876 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.476058 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924942-2a39-4029-b9c5-1e0662a5e876 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.476348 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924942-2a39-4029-b9c5-1e0662a5e876 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.477303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38924942-2a39-4029-b9c5-1e0662a5e876 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:21.477849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38924942-2a39-4029-b9c5-1e0662a5e876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:21.479837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38924942-2a39-4029-b9c5-1e0662a5e876 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:21.492210 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 26/50] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:21.498072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213899f-502b-4150-9f92-5a419e9691fa None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:21.604243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213899f-502b-4150-9f92-5a419e9691fa None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:21.605790 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:21.641118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede248d-28b9-4c4c-a169-2de00a5bf985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:21.641709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede248d-28b9-4c4c-a169-2de00a5bf985 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.642000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede248d-28b9-4c4c-a169-2de00a5bf985 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.642282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede248d-28b9-4c4c-a169-2de00a5bf985 None admin] PATH_INFO: `/v3/users/c44d1cb246ea41a89355002d9d2481af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.643471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede248d-28b9-4c4c-a169-2de00a5bf985 None admin] RBAC: Authorizing `identity:get_user(user_id=c44d1cb246ea41a89355002d9d2481af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:21.644963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede248d-28b9-4c4c-a169-2de00a5bf985 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:21.646594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 25/51] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:21 2022] GET /identity/v3/users/c44d1cb246ea41a89355002d9d2481af => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:21.652247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38924942-2a39-4029-b9c5-1e0662a5e876 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:21.702284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beade23c-e07b-45f1-814d-4acaa60ac449 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.702491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beade23c-e07b-45f1-814d-4acaa60ac449 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.703380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beade23c-e07b-45f1-814d-4acaa60ac449 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.704715 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 26/52] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:21.709818 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13230b-743e-4782-be4f-4e38ba4e6a33 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.710110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13230b-743e-4782-be4f-4e38ba4e6a33 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.710394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13230b-743e-4782-be4f-4e38ba4e6a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.729127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-5d13230b-743e-4782-be4f-4e38ba4e6a33 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:21.730343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5d13230b-743e-4782-be4f-4e38ba4e6a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:21.755856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38924942-2a39-4029-b9c5-1e0662a5e876 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:21.791143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2b2c7-4735-434a-8730-24ca8d96efb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:21.791788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2b2c7-4735-434a-8730-24ca8d96efb4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.792161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2b2c7-4735-434a-8730-24ca8d96efb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.792343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2b2c7-4735-434a-8730-24ca8d96efb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.793479 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2b2c7-4735-434a-8730-24ca8d96efb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:21.795071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2b2c7-4735-434a-8730-24ca8d96efb4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:21.796746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 27/53] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:21.798702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 27/54] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:29:21.802319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2b2c7-4735-434a-8730-24ca8d96efb4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:21.948064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:22.009171 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2b2c7-4735-434a-8730-24ca8d96efb4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:22.042534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec717618-6f19-483e-84c3-3446272e8e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:22.043158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec717618-6f19-483e-84c3-3446272e8e1d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:22.043448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec717618-6f19-483e-84c3-3446272e8e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:22.043730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec717618-6f19-483e-84c3-3446272e8e1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:22.044475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec717618-6f19-483e-84c3-3446272e8e1d None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.domain.id=default, user.id=c44d1cb246ea41a89355002d9d2481af) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:22.044793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec717618-6f19-483e-84c3-3446272e8e1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:22.046252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec717618-6f19-483e-84c3-3446272e8e1d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:22.055431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 28/55] 10.222.0.82 () {56 vars in 1322 bytes} [Wed Aug 24 03:29:21 2022] GET /identity/v3/role_assignments?user.id=c44d1cb246ea41a89355002d9d2481af&scope.domain.id=default&role.id=9e904216086e4a06abf9216b0c863f89 => generated 530 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:22.071209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13230b-743e-4782-be4f-4e38ba4e6a33 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:22.177797 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:22.180885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13230b-743e-4782-be4f-4e38ba4e6a33 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:22.223714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca9484-9297-4e2f-8cd3-e0472c0c74c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:22.224864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9484-9297-4e2f-8cd3-e0472c0c74c5 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:22.225397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9484-9297-4e2f-8cd3-e0472c0c74c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:22.225919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca9484-9297-4e2f-8cd3-e0472c0c74c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:22.228024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca9484-9297-4e2f-8cd3-e0472c0c74c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:22.230469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca9484-9297-4e2f-8cd3-e0472c0c74c5 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:22.233867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 28/56] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:22.240295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec717618-6f19-483e-84c3-3446272e8e1d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:22.339791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec717618-6f19-483e-84c3-3446272e8e1d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:22.341967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:22.373571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e93b89-107e-4145-8e43-5292ea55e70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:22.374183 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e93b89-107e-4145-8e43-5292ea55e70e None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:22.374473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e93b89-107e-4145-8e43-5292ea55e70e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:22.375430 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e93b89-107e-4145-8e43-5292ea55e70e None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:22.376511 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e93b89-107e-4145-8e43-5292ea55e70e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:22.377991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e93b89-107e-4145-8e43-5292ea55e70e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:22.464631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 29/57] 10.222.0.82 () {60 vars in 1140 bytes} [Wed Aug 24 03:29:22 2022] POST /identity/v3/projects => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:24.367935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0a034-b4cd-452d-af51-98b5ef9111cb None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:24.368428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0a034-b4cd-452d-af51-98b5ef9111cb None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:24.368428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0a034-b4cd-452d-af51-98b5ef9111cb None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:24.369742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 29/58] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:24.375119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7b13f-de84-443e-ac0a-84e27d444f10 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:24.375495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7b13f-de84-443e-ac0a-84e27d444f10 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:24.375835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7b13f-de84-443e-ac0a-84e27d444f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:24.394636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-17c7b13f-de84-443e-ac0a-84e27d444f10 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:24.395877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-17c7b13f-de84-443e-ac0a-84e27d444f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:24.461663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 30/59] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:24.742834 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0a034-b4cd-452d-af51-98b5ef9111cb None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:24.843001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0a034-b4cd-452d-af51-98b5ef9111cb None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:24.880857 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1881a-ae87-4c77-8273-0c5a857fd856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:24.881496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1881a-ae87-4c77-8273-0c5a857fd856 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:24.881786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1881a-ae87-4c77-8273-0c5a857fd856 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:24.882069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1881a-ae87-4c77-8273-0c5a857fd856 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:24.883262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1881a-ae87-4c77-8273-0c5a857fd856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:24.886326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1881a-ae87-4c77-8273-0c5a857fd856 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:24.887978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 30/60] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:24.894384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7b13f-de84-443e-ac0a-84e27d444f10 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:24.978204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:24.992791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7b13f-de84-443e-ac0a-84e27d444f10 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:25.027601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e38045-a620-44a7-9a0d-6712199393d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:25.028227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e38045-a620-44a7-9a0d-6712199393d8 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:25.028512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e38045-a620-44a7-9a0d-6712199393d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:25.029043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e38045-a620-44a7-9a0d-6712199393d8 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:25.029435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e38045-a620-44a7-9a0d-6712199393d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:25.030951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e38045-a620-44a7-9a0d-6712199393d8 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:25.164003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 31/61] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:29:24 2022] POST /identity/v3/users => generated 275 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:28.238096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedee733-75f6-4a19-9cce-dde61289438a None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:28.239083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedee733-75f6-4a19-9cce-dde61289438a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:28.239515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedee733-75f6-4a19-9cce-dde61289438a None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:28.241359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 31/62] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:28.247121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b4ebc-9e06-4fb2-9a52-1a659fe4aa14 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:28.248089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b4ebc-9e06-4fb2-9a52-1a659fe4aa14 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:28.248574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b4ebc-9e06-4fb2-9a52-1a659fe4aa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:28.278807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-8b6b4ebc-9e06-4fb2-9a52-1a659fe4aa14 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:28.280429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8b6b4ebc-9e06-4fb2-9a52-1a659fe4aa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:28.298567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f9161-4bc4-4026-8abc-2199b73feb73 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:28.299229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f9161-4bc4-4026-8abc-2199b73feb73 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:28.299836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f9161-4bc4-4026-8abc-2199b73feb73 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:28.301717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 32/63] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:28.307576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3976a-df95-4b6e-acb3-ab4048f1ea57 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:28.307862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3976a-df95-4b6e-acb3-ab4048f1ea57 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:28.308143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3976a-df95-4b6e-acb3-ab4048f1ea57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:28.343189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-95e3976a-df95-4b6e-acb3-ab4048f1ea57 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:28.346181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-95e3976a-df95-4b6e-acb3-ab4048f1ea57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:28.382992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:28.428068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 32/64] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:29:28.436040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 33/65] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:29:28.793738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fffb8-0875-4581-a59a-750c2e087a23 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:28.794109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fffb8-0875-4581-a59a-750c2e087a23 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:28.794353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fffb8-0875-4581-a59a-750c2e087a23 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:28.795760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 33/66] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:28.804572 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506e1f5-f198-4e17-9700-57d082e3ae42 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:28.804872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506e1f5-f198-4e17-9700-57d082e3ae42 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:28.805154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506e1f5-f198-4e17-9700-57d082e3ae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:28.825505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fffb8-0875-4581-a59a-750c2e087a23 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:28.836448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-0506e1f5-f198-4e17-9700-57d082e3ae42 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:28.837679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0506e1f5-f198-4e17-9700-57d082e3ae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:28.914330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:28.922306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 34/67] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:28.923247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506e1f5-f198-4e17-9700-57d082e3ae42 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:28.955405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fffb8-0875-4581-a59a-750c2e087a23 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:28.993854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8a837-31ec-430c-87a4-22557f3ccd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:28.994521 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8a837-31ec-430c-87a4-22557f3ccd9d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:28.994815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8a837-31ec-430c-87a4-22557f3ccd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:28.995188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8a837-31ec-430c-87a4-22557f3ccd9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.012238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8a837-31ec-430c-87a4-22557f3ccd9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-3ef8a837-31ec-430c-87a4-22557f3ccd9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:29.019435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:29.020339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:29.021642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 34/68] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:29:29.022378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2520c5-1e9a-4e14-9905-35fb61221d1c None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.022378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2520c5-1e9a-4e14-9905-35fb61221d1c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.022585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2520c5-1e9a-4e14-9905-35fb61221d1c None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.024101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 35/69] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:29.025273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1dd33b-45e8-4f18-87d5-855dc097199d None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.025566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1dd33b-45e8-4f18-87d5-855dc097199d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.025927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1dd33b-45e8-4f18-87d5-855dc097199d None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.027156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 36/70] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:29.028070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1dd33b-45e8-4f18-87d5-855dc097199d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.058805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:29.065406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506e1f5-f198-4e17-9700-57d082e3ae42 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.105633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54b4cc-41a1-45e7-892d-a2627bbbd3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.106763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54b4cc-41a1-45e7-892d-a2627bbbd3fa None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.107504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54b4cc-41a1-45e7-892d-a2627bbbd3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.107919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54b4cc-41a1-45e7-892d-a2627bbbd3fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.119355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54b4cc-41a1-45e7-892d-a2627bbbd3fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-1b54b4cc-41a1-45e7-892d-a2627bbbd3fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:29.125881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:29.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:29.128119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:29.130722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 35/71] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:29:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:29.132755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df99f78-f1f7-49c1-b933-6223863e341d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.133203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df99f78-f1f7-49c1-b933-6223863e341d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.133636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df99f78-f1f7-49c1-b933-6223863e341d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.152915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2df99f78-f1f7-49c1-b933-6223863e341d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:29.153903 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1dd33b-45e8-4f18-87d5-855dc097199d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.154168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2df99f78-f1f7-49c1-b933-6223863e341d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:29.189364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:29.195045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb33a1-f1e0-497f-8654-cb35eb75eaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.195709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb33a1-f1e0-497f-8654-cb35eb75eaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.195993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb33a1-f1e0-497f-8654-cb35eb75eaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.196268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb33a1-f1e0-497f-8654-cb35eb75eaa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.196952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb33a1-f1e0-497f-8654-cb35eb75eaa4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:29.197261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb33a1-f1e0-497f-8654-cb35eb75eaa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:29.198712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb33a1-f1e0-497f-8654-cb35eb75eaa4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:29.208344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 37/72] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:29.209639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7881075-8273-4435-ad12-b34d5ee03644 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.209938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7881075-8273-4435-ad12-b34d5ee03644 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.210254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7881075-8273-4435-ad12-b34d5ee03644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.228794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e7881075-8273-4435-ad12-b34d5ee03644 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:29.230111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e7881075-8273-4435-ad12-b34d5ee03644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:29.230750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 36/73] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:29.231434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df99f78-f1f7-49c1-b933-6223863e341d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.314617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 38/74] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:29.314617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7881075-8273-4435-ad12-b34d5ee03644 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.329804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:29.351200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df99f78-f1f7-49c1-b933-6223863e341d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.409774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c8b9c-1ba9-4690-b3a8-bb4669cc6eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.410383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c8b9c-1ba9-4690-b3a8-bb4669cc6eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.410718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c8b9c-1ba9-4690-b3a8-bb4669cc6eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.410997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c8b9c-1ba9-4690-b3a8-bb4669cc6eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.411678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c8b9c-1ba9-4690-b3a8-bb4669cc6eb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:29.411986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c8b9c-1ba9-4690-b3a8-bb4669cc6eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:29.413418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c8b9c-1ba9-4690-b3a8-bb4669cc6eb7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:29.423062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 37/75] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:29.428168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c8b9c-1ba9-4690-b3a8-bb4669cc6eb7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.439526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7881075-8273-4435-ad12-b34d5ee03644 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.480015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:29.495618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c7d9e-23da-4917-a693-c32493fc9a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.496757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c7d9e-23da-4917-a693-c32493fc9a9b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.497182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c7d9e-23da-4917-a693-c32493fc9a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.498243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c7d9e-23da-4917-a693-c32493fc9a9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.510106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c7d9e-23da-4917-a693-c32493fc9a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-209c7d9e-23da-4917-a693-c32493fc9a9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:29.515060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:29.516895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:29.517767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:29.525444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 39/76] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:29.525444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c7d9e-23da-4917-a693-c32493fc9a9b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.543086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c8b9c-1ba9-4690-b3a8-bb4669cc6eb7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.577523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23c59d-7468-4013-9384-679495611c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.578849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23c59d-7468-4013-9384-679495611c48 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.579635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23c59d-7468-4013-9384-679495611c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.579635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23c59d-7468-4013-9384-679495611c48 None admin] PATH_INFO: `/v3/users/3eb5ec627dc1453ca7e211a5b47289af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.594488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23c59d-7468-4013-9384-679495611c48 None admin] RBAC: Authorizing `identity:get_user(user_id=3eb5ec627dc1453ca7e211a5b47289af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:29.595998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23c59d-7468-4013-9384-679495611c48 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:29.597676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 38/77] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/users/3eb5ec627dc1453ca7e211a5b47289af => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:29.598449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23c59d-7468-4013-9384-679495611c48 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.607591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:29.635877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c7d9e-23da-4917-a693-c32493fc9a9b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.675049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d6e9a-f597-4ab6-9516-7b8acfd5eaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.675759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d6e9a-f597-4ab6-9516-7b8acfd5eaea None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.676088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d6e9a-f597-4ab6-9516-7b8acfd5eaea None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.676365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d6e9a-f597-4ab6-9516-7b8acfd5eaea None admin] PATH_INFO: `/v3/users/c44d1cb246ea41a89355002d9d2481af` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.677848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d6e9a-f597-4ab6-9516-7b8acfd5eaea None admin] RBAC: Authorizing `identity:get_user(user_id=c44d1cb246ea41a89355002d9d2481af)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:29.679352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d6e9a-f597-4ab6-9516-7b8acfd5eaea None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:29.680940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 40/78] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/users/c44d1cb246ea41a89355002d9d2481af => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:29.681701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d6e9a-f597-4ab6-9516-7b8acfd5eaea None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.701772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23c59d-7468-4013-9384-679495611c48 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.737820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c618d-b80f-4fe6-be10-2901621ffab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.738467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c618d-b80f-4fe6-be10-2901621ffab9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.738937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:29.739192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c618d-b80f-4fe6-be10-2901621ffab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.739488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c618d-b80f-4fe6-be10-2901621ffab9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.749639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c618d-b80f-4fe6-be10-2901621ffab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-173c618d-b80f-4fe6-be10-2901621ffab9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:29.755219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:29.756115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:29.756890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 39/79] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:29.757552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c618d-b80f-4fe6-be10-2901621ffab9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.788630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d6e9a-f597-4ab6-9516-7b8acfd5eaea None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.822805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:29.823012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3935f09-754c-4478-b218-5684c5234477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.823556 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3935f09-754c-4478-b218-5684c5234477 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.823848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3935f09-754c-4478-b218-5684c5234477 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.824136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3935f09-754c-4478-b218-5684c5234477 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.824812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3935f09-754c-4478-b218-5684c5234477 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:29.825124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3935f09-754c-4478-b218-5684c5234477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:29.826550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3935f09-754c-4478-b218-5684c5234477 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:29.839427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 41/80] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:29.839427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3935f09-754c-4478-b218-5684c5234477 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.858406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c618d-b80f-4fe6-be10-2901621ffab9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.894370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2047a-eee9-4c83-abd8-ed615685c658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.895075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2047a-eee9-4c83-abd8-ed615685c658 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.895368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2047a-eee9-4c83-abd8-ed615685c658 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.895650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2047a-eee9-4c83-abd8-ed615685c658 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.896819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2047a-eee9-4c83-abd8-ed615685c658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:29.898305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2047a-eee9-4c83-abd8-ed615685c658 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:29.899953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 40/81] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:29.904526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2047a-eee9-4c83-abd8-ed615685c658 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:29.942755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3935f09-754c-4478-b218-5684c5234477 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.956425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:29.977326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871142e-f546-4a91-aa9b-f21a24d45d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.977943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871142e-f546-4a91-aa9b-f21a24d45d0c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.978230 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871142e-f546-4a91-aa9b-f21a24d45d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.978528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871142e-f546-4a91-aa9b-f21a24d45d0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.008828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2047a-eee9-4c83-abd8-ed615685c658 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:30.044513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a18cb-7750-4eca-9210-5d458e108450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:30.045137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a18cb-7750-4eca-9210-5d458e108450 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:30.045423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a18cb-7750-4eca-9210-5d458e108450 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:30.045702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a18cb-7750-4eca-9210-5d458e108450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.048988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:30.050704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871142e-f546-4a91-aa9b-f21a24d45d0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.052208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871142e-f546-4a91-aa9b-f21a24d45d0c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:30.116563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a18cb-7750-4eca-9210-5d458e108450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.117998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a18cb-7750-4eca-9210-5d458e108450 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:30.151488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 42/82] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:30.152346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871142e-f546-4a91-aa9b-f21a24d45d0c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:30.167549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:30.217662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 41/83] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:30.219052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a18cb-7750-4eca-9210-5d458e108450 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:30.252470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:30.252959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871142e-f546-4a91-aa9b-f21a24d45d0c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:30.287126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d0f06-3bae-4563-a84f-4bd61883ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:30.287740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d0f06-3bae-4563-a84f-4bd61883ed7b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:30.288033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d0f06-3bae-4563-a84f-4bd61883ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:30.288318 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d0f06-3bae-4563-a84f-4bd61883ed7b None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.288987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d0f06-3bae-4563-a84f-4bd61883ed7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:30.289302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d0f06-3bae-4563-a84f-4bd61883ed7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.290734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d0f06-3bae-4563-a84f-4bd61883ed7b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:30.299645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 43/84] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:30.300205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d0f06-3bae-4563-a84f-4bd61883ed7b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:30.323954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a18cb-7750-4eca-9210-5d458e108450 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:30.360395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634783a9-5561-47ef-abdf-d2591c170a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:30.361057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634783a9-5561-47ef-abdf-d2591c170a7a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:30.361313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634783a9-5561-47ef-abdf-d2591c170a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:30.361589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634783a9-5561-47ef-abdf-d2591c170a7a None admin] PATH_INFO: `/v3/users/3eb5ec627dc1453ca7e211a5b47289af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.362754 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634783a9-5561-47ef-abdf-d2591c170a7a None admin] RBAC: Authorizing `identity:get_user(user_id=3eb5ec627dc1453ca7e211a5b47289af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.364214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634783a9-5561-47ef-abdf-d2591c170a7a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:30.365792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 42/85] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:30 2022] GET /identity/v3/users/3eb5ec627dc1453ca7e211a5b47289af => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:30.366717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634783a9-5561-47ef-abdf-d2591c170a7a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:30.390929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:30.413925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d0f06-3bae-4563-a84f-4bd61883ed7b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:30.456842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58430cb6-7166-4db4-a76d-3942357f198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:30.457484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58430cb6-7166-4db4-a76d-3942357f198e None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:30.457780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58430cb6-7166-4db4-a76d-3942357f198e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:30.458050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58430cb6-7166-4db4-a76d-3942357f198e None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.458775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58430cb6-7166-4db4-a76d-3942357f198e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.460431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58430cb6-7166-4db4-a76d-3942357f198e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:30.467669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634783a9-5561-47ef-abdf-d2591c170a7a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:30.469977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:30.503374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdc449-37d6-4abb-8a5d-f08e70f9edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:30.504014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdc449-37d6-4abb-8a5d-f08e70f9edb5 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:30.504304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdc449-37d6-4abb-8a5d-f08e70f9edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:30.504584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdc449-37d6-4abb-8a5d-f08e70f9edb5 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.505745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdc449-37d6-4abb-8a5d-f08e70f9edb5 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.507321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdc449-37d6-4abb-8a5d-f08e70f9edb5 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:30.508894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 43/86] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:30 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:30.509665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdc449-37d6-4abb-8a5d-f08e70f9edb5 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:30.552337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 44/87] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:29:30 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:30.553407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58430cb6-7166-4db4-a76d-3942357f198e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:30.622691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdc449-37d6-4abb-8a5d-f08e70f9edb5 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:30.666282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d6e87-1ba8-4190-9977-7b923fa58e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:30.667078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d6e87-1ba8-4190-9977-7b923fa58e13 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:30.667377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d6e87-1ba8-4190-9977-7b923fa58e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:30.667783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d6e87-1ba8-4190-9977-7b923fa58e13 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.669072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d6e87-1ba8-4190-9977-7b923fa58e13 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.670653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d6e87-1ba8-4190-9977-7b923fa58e13 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:30.672689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 44/88] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:30 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:30.678543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d6e87-1ba8-4190-9977-7b923fa58e13 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:30.691948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58430cb6-7166-4db4-a76d-3942357f198e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:30.725942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:30.727130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:30.731931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481afb3-f9a1-4156-9257-08d5ae7f70e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:30.732917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481afb3-f9a1-4156-9257-08d5ae7f70e4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:30.733122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481afb3-f9a1-4156-9257-08d5ae7f70e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:30.733896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481afb3-f9a1-4156-9257-08d5ae7f70e4 None admin] PATH_INFO: `/v3/users/3eb5ec627dc1453ca7e211a5b47289af` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.739053 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481afb3-f9a1-4156-9257-08d5ae7f70e4 None admin] RBAC: Authorizing `identity:get_user(user_id=3eb5ec627dc1453ca7e211a5b47289af)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.739053 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481afb3-f9a1-4156-9257-08d5ae7f70e4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:30.739053 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 45/89] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:30 2022] GET /identity/v3/users/3eb5ec627dc1453ca7e211a5b47289af => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:30.739814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481afb3-f9a1-4156-9257-08d5ae7f70e4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:30.816636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d6e87-1ba8-4190-9977-7b923fa58e13 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:30.862265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8662b7e-78fe-4355-a12b-c3664a3ac72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:30.862935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8662b7e-78fe-4355-a12b-c3664a3ac72e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:30.863228 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8662b7e-78fe-4355-a12b-c3664a3ac72e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:30.863564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8662b7e-78fe-4355-a12b-c3664a3ac72e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.875320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481afb3-f9a1-4156-9257-08d5ae7f70e4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:30.920061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf110b2-dd7d-4ef9-aad9-60179a3931c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:30.920690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf110b2-dd7d-4ef9-aad9-60179a3931c0 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:30.920980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf110b2-dd7d-4ef9-aad9-60179a3931c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:30.921260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf110b2-dd7d-4ef9-aad9-60179a3931c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:30.922145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf110b2-dd7d-4ef9-aad9-60179a3931c0 None admin] RBAC: Adding query filter params (role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=86e412e829cf47228db2532cb4c7c4e4, user.id=3eb5ec627dc1453ca7e211a5b47289af) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:30.922789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf110b2-dd7d-4ef9-aad9-60179a3931c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.924209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf110b2-dd7d-4ef9-aad9-60179a3931c0 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:30.930711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 46/90] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:30 2022] GET /identity/v3/role_assignments?user.id=3eb5ec627dc1453ca7e211a5b47289af&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:30.931715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf110b2-dd7d-4ef9-aad9-60179a3931c0 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:30.955962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8662b7e-78fe-4355-a12b-c3664a3ac72e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:30.958015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8662b7e-78fe-4355-a12b-c3664a3ac72e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:31.006984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:31.050690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf110b2-dd7d-4ef9-aad9-60179a3931c0 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:31.091594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ecb9f-df3d-4524-b033-d0628a5a51fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:31.092232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ecb9f-df3d-4524-b033-d0628a5a51fe None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:31.092521 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ecb9f-df3d-4524-b033-d0628a5a51fe None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:31.092801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ecb9f-df3d-4524-b033-d0628a5a51fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:31.093678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ecb9f-df3d-4524-b033-d0628a5a51fe None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=86e412e829cf47228db2532cb4c7c4e4, user.id=c44d1cb246ea41a89355002d9d2481af) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:31.094061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ecb9f-df3d-4524-b033-d0628a5a51fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:31.095898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ecb9f-df3d-4524-b033-d0628a5a51fe None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:31.102616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 47/91] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:30 2022] GET /identity/v3/role_assignments?user.id=c44d1cb246ea41a89355002d9d2481af&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=9e904216086e4a06abf9216b0c863f89 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:31.103559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ecb9f-df3d-4524-b033-d0628a5a51fe None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:31.110976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:31.231130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 45/92] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:31.240569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8662b7e-78fe-4355-a12b-c3664a3ac72e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:31.260793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ecb9f-df3d-4524-b033-d0628a5a51fe None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:31.329955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:31.359803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55427b-b3b0-48d1-b103-bacf35eac714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:31.360914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55427b-b3b0-48d1-b103-bacf35eac714 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:31.361216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55427b-b3b0-48d1-b103-bacf35eac714 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:31.361506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55427b-b3b0-48d1-b103-bacf35eac714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:31.412726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8662b7e-78fe-4355-a12b-c3664a3ac72e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:31.480344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a87ad3-8656-4a4d-a33e-1203a06cf4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:31.481428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a87ad3-8656-4a4d-a33e-1203a06cf4de None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:31.481998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a87ad3-8656-4a4d-a33e-1203a06cf4de None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:31.482766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a87ad3-8656-4a4d-a33e-1203a06cf4de None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:31.487716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a87ad3-8656-4a4d-a33e-1203a06cf4de None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:31.490222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a87ad3-8656-4a4d-a33e-1203a06cf4de None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:31.493706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 46/93] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:31 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:31.501120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a87ad3-8656-4a4d-a33e-1203a06cf4de None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:31.511383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55427b-b3b0-48d1-b103-bacf35eac714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:31.512919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55427b-b3b0-48d1-b103-bacf35eac714 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:31.518368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:31.708815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a87ad3-8656-4a4d-a33e-1203a06cf4de None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:31.710972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 48/94] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:31.717256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55427b-b3b0-48d1-b103-bacf35eac714 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:31.790972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:31.799516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233aee3-0a90-4531-9c44-7629ceb9139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:31.800150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233aee3-0a90-4531-9c44-7629ceb9139b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:31.800444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233aee3-0a90-4531-9c44-7629ceb9139b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:31.800728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233aee3-0a90-4531-9c44-7629ceb9139b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:31.801483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233aee3-0a90-4531-9c44-7629ceb9139b None admin] RBAC: Adding query filter params (role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=86e412e829cf47228db2532cb4c7c4e4, user.id=3eb5ec627dc1453ca7e211a5b47289af) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:31.801804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233aee3-0a90-4531-9c44-7629ceb9139b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:31.803294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233aee3-0a90-4531-9c44-7629ceb9139b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:31.822242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 47/95] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:31 2022] GET /identity/v3/role_assignments?user.id=3eb5ec627dc1453ca7e211a5b47289af&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:31.887932 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55427b-b3b0-48d1-b103-bacf35eac714 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:31.928936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74669492-633a-4421-84d8-8e00aabd62f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:31.929622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74669492-633a-4421-84d8-8e00aabd62f1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:31.929912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74669492-633a-4421-84d8-8e00aabd62f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:31.930264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74669492-633a-4421-84d8-8e00aabd62f1 None admin] PATH_INFO: `/v3/projects/398402efcd674cf48c4694271975d2b1` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:31.931632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74669492-633a-4421-84d8-8e00aabd62f1 None admin] RBAC: Authorizing `identity:get_project(project_id=398402efcd674cf48c4694271975d2b1)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:31.933276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74669492-633a-4421-84d8-8e00aabd62f1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:31.935056 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 49/96] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:31 2022] GET /identity/v3/projects/398402efcd674cf48c4694271975d2b1 => generated 312 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:31.948163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233aee3-0a90-4531-9c44-7629ceb9139b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:32.103540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:32.161482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233aee3-0a90-4531-9c44-7629ceb9139b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:32.221394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da534a-3861-4054-b5fd-e2b614a96c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:32.222028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da534a-3861-4054-b5fd-e2b614a96c79 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:32.222320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da534a-3861-4054-b5fd-e2b614a96c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:32.222630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da534a-3861-4054-b5fd-e2b614a96c79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:32.223389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da534a-3861-4054-b5fd-e2b614a96c79 None admin] RBAC: Adding query filter params (role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=398402efcd674cf48c4694271975d2b1, user.id=3eb5ec627dc1453ca7e211a5b47289af) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:32.223711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da534a-3861-4054-b5fd-e2b614a96c79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:32.225275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da534a-3861-4054-b5fd-e2b614a96c79 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:32.236710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 48/97] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:31 2022] GET /identity/v3/role_assignments?user.id=3eb5ec627dc1453ca7e211a5b47289af&scope.project.id=398402efcd674cf48c4694271975d2b1&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:33.501474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831f0f-6193-4fa2-9ac3-c8a43307a689 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:33.501986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831f0f-6193-4fa2-9ac3-c8a43307a689 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:33.502060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831f0f-6193-4fa2-9ac3-c8a43307a689 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:33.503763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 50/98] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:33.509709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ea91a-e762-447c-bc0b-beb29e356f18 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:33.510209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ea91a-e762-447c-bc0b-beb29e356f18 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:33.510732 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ea91a-e762-447c-bc0b-beb29e356f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:33.544506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-531ea91a-e762-447c-bc0b-beb29e356f18 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:33.546600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-531ea91a-e762-447c-bc0b-beb29e356f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:33.649138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 49/99] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:34.005647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927edf03-2086-4a09-b340-4698ac68fd0e None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.005941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927edf03-2086-4a09-b340-4698ac68fd0e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.006422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927edf03-2086-4a09-b340-4698ac68fd0e None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.008102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 51/100] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:34.019152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5fd12-fd2e-470a-b1b1-34cf7ceafc2e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.020337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5fd12-fd2e-470a-b1b1-34cf7ceafc2e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.020818 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5fd12-fd2e-470a-b1b1-34cf7ceafc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.025436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0fc8-7343-4005-a866-a2a22977540a None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.026123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0fc8-7343-4005-a866-a2a22977540a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.026666 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0fc8-7343-4005-a866-a2a22977540a None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.028237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 52/101] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:34.034499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff886dbd-3c6e-40c1-bd3c-4d921c45a5de None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.035106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff886dbd-3c6e-40c1-bd3c-4d921c45a5de None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.035618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff886dbd-3c6e-40c1-bd3c-4d921c45a5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.049277 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-3fd5fd12-fd2e-470a-b1b1-34cf7ceafc2e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:34.050215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3fd5fd12-fd2e-470a-b1b1-34cf7ceafc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:34.056098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ff886dbd-3c6e-40c1-bd3c-4d921c45a5de None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:34.057589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ff886dbd-3c6e-40c1-bd3c-4d921c45a5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:34.156011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 53/102] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:29:34.183078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:34.197565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 50/103] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:34.268865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff886dbd-3c6e-40c1-bd3c-4d921c45a5de None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:34.314536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:34.356184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eec0ba-c31d-41d9-879a-f96245739a9b None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.356532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eec0ba-c31d-41d9-879a-f96245739a9b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.356753 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eec0ba-c31d-41d9-879a-f96245739a9b None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.358547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 51/104] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:34.367999 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b32e4c-935c-4b8a-a653-d1f951ce87b4 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.368305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b32e4c-935c-4b8a-a653-d1f951ce87b4 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.368572 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b32e4c-935c-4b8a-a653-d1f951ce87b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.404166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b0b32e4c-935c-4b8a-a653-d1f951ce87b4 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:34.405409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b0b32e4c-935c-4b8a-a653-d1f951ce87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:34.422672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff886dbd-3c6e-40c1-bd3c-4d921c45a5de None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:34.466520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819e80d-cfdf-4e2d-bc88-625c33149137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:34.467487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819e80d-cfdf-4e2d-bc88-625c33149137 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.467800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819e80d-cfdf-4e2d-bc88-625c33149137 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.468087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819e80d-cfdf-4e2d-bc88-625c33149137 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.470240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819e80d-cfdf-4e2d-bc88-625c33149137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:34.471904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e819e80d-cfdf-4e2d-bc88-625c33149137 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:34.473709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 54/105] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:34.474947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5b410-f106-4385-8309-d1cf410438d9 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.475247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5b410-f106-4385-8309-d1cf410438d9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.475532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5b410-f106-4385-8309-d1cf410438d9 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.476985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 55/106] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:34.489663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f92b0-baea-4929-a4d3-a2e29f233785 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.489952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f92b0-baea-4929-a4d3-a2e29f233785 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.490233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f92b0-baea-4929-a4d3-a2e29f233785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.540295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 52/107] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:34.540924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b32e4c-935c-4b8a-a653-d1f951ce87b4 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:34.542537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a61f92b0-baea-4929-a4d3-a2e29f233785 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:34.543949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a61f92b0-baea-4929-a4d3-a2e29f233785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:34.687599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 56/108] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:34.688857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f92b0-baea-4929-a4d3-a2e29f233785 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:34.700189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b32e4c-935c-4b8a-a653-d1f951ce87b4 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:34.726975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:34.739619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc479-7307-4511-abb0-268e907498b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:34.744192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc479-7307-4511-abb0-268e907498b9 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:34.744192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc479-7307-4511-abb0-268e907498b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:34.744192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc479-7307-4511-abb0-268e907498b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:34.744192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fc479-7307-4511-abb0-268e907498b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:34.744192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fc479-7307-4511-abb0-268e907498b9 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:34.755985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:34.860172 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 53/109] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:29:34 2022] POST /identity/v3/projects => generated 302 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:34.862321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc479-7307-4511-abb0-268e907498b9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:34.886632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f92b0-baea-4929-a4d3-a2e29f233785 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:35.060031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9443a5-0a02-4086-b8d9-231dfc2f2d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:35.061105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9443a5-0a02-4086-b8d9-231dfc2f2d54 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.061626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9443a5-0a02-4086-b8d9-231dfc2f2d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.062115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9443a5-0a02-4086-b8d9-231dfc2f2d54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.101487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9443a5-0a02-4086-b8d9-231dfc2f2d54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:35.106385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc479-7307-4511-abb0-268e907498b9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-0d9443a5-0a02-4086-b8d9-231dfc2f2d54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:35.106579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:35.107706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:35.108547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:35.109844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 57/110] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:35.110805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9443a5-0a02-4086-b8d9-231dfc2f2d54 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:35.180925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2295e-4c2e-4841-aeb9-88b10087eca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:35.183140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2295e-4c2e-4841-aeb9-88b10087eca6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.183140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2295e-4c2e-4841-aeb9-88b10087eca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.187174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2295e-4c2e-4841-aeb9-88b10087eca6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.205297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2295e-4c2e-4841-aeb9-88b10087eca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-cfd2295e-4c2e-4841-aeb9-88b10087eca6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:35.218683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:35.219928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:29:35.220743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:35.223188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 54/111] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:35.224189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2295e-4c2e-4841-aeb9-88b10087eca6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:35.257629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:35.279345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9443a5-0a02-4086-b8d9-231dfc2f2d54 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:35.287659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:35.327809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fae22-576c-4c1e-bcf9-f838fde1365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:35.328330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fae22-576c-4c1e-bcf9-f838fde1365d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.328590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fae22-576c-4c1e-bcf9-f838fde1365d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.328870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fae22-576c-4c1e-bcf9-f838fde1365d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.343248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974fae22-576c-4c1e-bcf9-f838fde1365d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-974fae22-576c-4c1e-bcf9-f838fde1365d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:35.351235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:35.352163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:35.352981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:35.355863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 58/112] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:29:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:35.360973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fae22-576c-4c1e-bcf9-f838fde1365d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:35.388054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2295e-4c2e-4841-aeb9-88b10087eca6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:35.457939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e0709-0b83-474e-b20c-79c22703cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:35.458991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e0709-0b83-474e-b20c-79c22703cca8 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.459515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e0709-0b83-474e-b20c-79c22703cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.460012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e0709-0b83-474e-b20c-79c22703cca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.460913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e0709-0b83-474e-b20c-79c22703cca8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:35.461439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e0709-0b83-474e-b20c-79c22703cca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:35.463295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e0709-0b83-474e-b20c-79c22703cca8 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:35.475536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 55/113] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:35.476754 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e0709-0b83-474e-b20c-79c22703cca8 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:35.512609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fae22-576c-4c1e-bcf9-f838fde1365d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:35.559182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd7533-0872-460e-bceb-f74057228961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:35.559811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd7533-0872-460e-bceb-f74057228961 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.560105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd7533-0872-460e-bceb-f74057228961 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.560388 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd7533-0872-460e-bceb-f74057228961 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.574560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd7533-0872-460e-bceb-f74057228961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-17fd7533-0872-460e-bceb-f74057228961 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:35.582565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:35.583756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:35.584559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 59/114] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:29:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:35.585751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd7533-0872-460e-bceb-f74057228961 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:35.591326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:35.615074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:35.620760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e0709-0b83-474e-b20c-79c22703cca8 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:35.659325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca61ddf-2f74-4e8e-817b-5a266ffed002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:35.659955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca61ddf-2f74-4e8e-817b-5a266ffed002 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.660245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca61ddf-2f74-4e8e-817b-5a266ffed002 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.660522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca61ddf-2f74-4e8e-817b-5a266ffed002 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.661203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca61ddf-2f74-4e8e-817b-5a266ffed002 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:35.661513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca61ddf-2f74-4e8e-817b-5a266ffed002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:35.663033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca61ddf-2f74-4e8e-817b-5a266ffed002 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:35.678763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 56/115] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:35.678763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca61ddf-2f74-4e8e-817b-5a266ffed002 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:35.708016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd7533-0872-460e-bceb-f74057228961 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:35.748933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da45002-1775-4133-a33a-647b252a037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:35.749855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da45002-1775-4133-a33a-647b252a037e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.750695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da45002-1775-4133-a33a-647b252a037e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.751193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da45002-1775-4133-a33a-647b252a037e None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.752159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da45002-1775-4133-a33a-647b252a037e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:35.752479 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da45002-1775-4133-a33a-647b252a037e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:35.753920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da45002-1775-4133-a33a-647b252a037e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:35.766399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 60/116] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:35.766399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94451048-27ab-4f89-8d1c-a0bae3ce77e7 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.766399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94451048-27ab-4f89-8d1c-a0bae3ce77e7 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.766399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94451048-27ab-4f89-8d1c-a0bae3ce77e7 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.770540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 61/117] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:35.770540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94451048-27ab-4f89-8d1c-a0bae3ce77e7 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:35.796578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca61ddf-2f74-4e8e-817b-5a266ffed002 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:35.841628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1017cb4-7069-4afb-a694-bf6240cebaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:35.843589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1017cb4-7069-4afb-a694-bf6240cebaf5 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.843589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1017cb4-7069-4afb-a694-bf6240cebaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.843589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1017cb4-7069-4afb-a694-bf6240cebaf5 None admin] PATH_INFO: `/v3/users/3eb5ec627dc1453ca7e211a5b47289af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.846055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1017cb4-7069-4afb-a694-bf6240cebaf5 None admin] RBAC: Authorizing `identity:get_user(user_id=3eb5ec627dc1453ca7e211a5b47289af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:35.847587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1017cb4-7069-4afb-a694-bf6240cebaf5 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:35.849249 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 57/118] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:35 2022] GET /identity/v3/users/3eb5ec627dc1453ca7e211a5b47289af => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:35.850040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1017cb4-7069-4afb-a694-bf6240cebaf5 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:35.863541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:35.893835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:35.923923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94451048-27ab-4f89-8d1c-a0bae3ce77e7 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:35.967808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734831dc-64ed-4bc3-994d-16c36259778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:35.968444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734831dc-64ed-4bc3-994d-16c36259778f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:35.969297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734831dc-64ed-4bc3-994d-16c36259778f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:35.970088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734831dc-64ed-4bc3-994d-16c36259778f None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:35.970265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734831dc-64ed-4bc3-994d-16c36259778f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:35.970626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734831dc-64ed-4bc3-994d-16c36259778f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:35.972099 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734831dc-64ed-4bc3-994d-16c36259778f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:35.982627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 62/119] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:29:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:35.983485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734831dc-64ed-4bc3-994d-16c36259778f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:36.008236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1017cb4-7069-4afb-a694-bf6240cebaf5 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:36.048723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4c39a-798d-4453-a38e-058fe4da61db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:36.049486 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4c39a-798d-4453-a38e-058fe4da61db None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:36.049848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4c39a-798d-4453-a38e-058fe4da61db None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:36.050207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4c39a-798d-4453-a38e-058fe4da61db None admin] PATH_INFO: `/v3/users/9565cb05f1204af6af9e9f4780d8cf0b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:36.074635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4c39a-798d-4453-a38e-058fe4da61db None admin] RBAC: Authorizing `identity:get_user(user_id=9565cb05f1204af6af9e9f4780d8cf0b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:36.079354 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4c39a-798d-4453-a38e-058fe4da61db None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:36.086290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 58/120] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:35 2022] GET /identity/v3/users/9565cb05f1204af6af9e9f4780d8cf0b => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:36.086290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c4ed9-eae8-456e-a62d-bcb0f3ff5079 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:36.086290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c4ed9-eae8-456e-a62d-bcb0f3ff5079 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:36.086290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c4ed9-eae8-456e-a62d-bcb0f3ff5079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:36.109709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-939c4ed9-eae8-456e-a62d-bcb0f3ff5079 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:36.110987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-939c4ed9-eae8-456e-a62d-bcb0f3ff5079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:36.119236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734831dc-64ed-4bc3-994d-16c36259778f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:36.152919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:36.162196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbe6a7-4f72-48f7-86ee-035385fd9eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:36.162930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbe6a7-4f72-48f7-86ee-035385fd9eda None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:36.163218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbe6a7-4f72-48f7-86ee-035385fd9eda None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:36.173520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbe6a7-4f72-48f7-86ee-035385fd9eda None admin] PATH_INFO: `/v3/users/c44d1cb246ea41a89355002d9d2481af` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:36.173520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbe6a7-4f72-48f7-86ee-035385fd9eda None admin] RBAC: Authorizing `identity:get_user(user_id=c44d1cb246ea41a89355002d9d2481af)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:36.173520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbe6a7-4f72-48f7-86ee-035385fd9eda None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:36.173520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 63/121] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:35 2022] GET /identity/v3/users/c44d1cb246ea41a89355002d9d2481af => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:36.173520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbe6a7-4f72-48f7-86ee-035385fd9eda None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:36.200874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 59/122] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:36.201891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c4ed9-eae8-456e-a62d-bcb0f3ff5079 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:36.229096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:36.345627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbe6a7-4f72-48f7-86ee-035385fd9eda None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:36.355422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c4ed9-eae8-456e-a62d-bcb0f3ff5079 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:36.405031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d146c-5880-4793-b2c4-7ad9f223fff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:36.407193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d146c-5880-4793-b2c4-7ad9f223fff6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:36.407983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d146c-5880-4793-b2c4-7ad9f223fff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:36.408596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d146c-5880-4793-b2c4-7ad9f223fff6 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:36.410446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d146c-5880-4793-b2c4-7ad9f223fff6 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:36.411902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222812e-6f22-412a-87e7-5620473e8e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:36.412698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d146c-5880-4793-b2c4-7ad9f223fff6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:36.413010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222812e-6f22-412a-87e7-5620473e8e8d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:36.413419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222812e-6f22-412a-87e7-5620473e8e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:36.413771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222812e-6f22-412a-87e7-5620473e8e8d None admin] PATH_INFO: `/v3/users/3eb5ec627dc1453ca7e211a5b47289af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:36.415126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222812e-6f22-412a-87e7-5620473e8e8d None admin] RBAC: Authorizing `identity:get_user(user_id=3eb5ec627dc1453ca7e211a5b47289af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:36.415584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 64/123] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:36 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:36.416617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d146c-5880-4793-b2c4-7ad9f223fff6 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:36.416718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222812e-6f22-412a-87e7-5620473e8e8d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:36.422637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 60/124] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:36 2022] GET /identity/v3/users/3eb5ec627dc1453ca7e211a5b47289af => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:36.422637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222812e-6f22-412a-87e7-5620473e8e8d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:36.494723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:36.545204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d146c-5880-4793-b2c4-7ad9f223fff6 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:36.549163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:36.575324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222812e-6f22-412a-87e7-5620473e8e8d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:36.599711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd8d37-6634-449d-bdf1-bace8b20d120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:36.600867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd8d37-6634-449d-bdf1-bace8b20d120 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:36.601352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd8d37-6634-449d-bdf1-bace8b20d120 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:36.601835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd8d37-6634-449d-bdf1-bace8b20d120 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:36.634737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37353d0b-1c06-4d7c-bae5-504d4a201e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:36.636383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37353d0b-1c06-4d7c-bae5-504d4a201e15 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:36.636383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37353d0b-1c06-4d7c-bae5-504d4a201e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:36.636801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37353d0b-1c06-4d7c-bae5-504d4a201e15 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:36.638287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37353d0b-1c06-4d7c-bae5-504d4a201e15 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:36.640253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37353d0b-1c06-4d7c-bae5-504d4a201e15 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:36.642929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 61/125] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:36 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:36.643806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37353d0b-1c06-4d7c-bae5-504d4a201e15 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:36.702634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd8d37-6634-449d-bdf1-bace8b20d120 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:36.704127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd8d37-6634-449d-bdf1-bace8b20d120 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:36.784214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37353d0b-1c06-4d7c-bae5-504d4a201e15 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:36.786855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:36.830390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 65/126] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:36.832059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd8d37-6634-449d-bdf1-bace8b20d120 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:36.846544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dcab2-b5b9-4962-8560-e3271028ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:36.847535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dcab2-b5b9-4962-8560-e3271028ec05 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:36.848009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dcab2-b5b9-4962-8560-e3271028ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:36.848506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dcab2-b5b9-4962-8560-e3271028ec05 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4/users/3eb5ec627dc1453ca7e211a5b47289af/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:36.851017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dcab2-b5b9-4962-8560-e3271028ec05 None admin] RBAC: Authorizing `identity:create_grant(project_id=86e412e829cf47228db2532cb4c7c4e4, user_id=3eb5ec627dc1453ca7e211a5b47289af, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:36.852514 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dcab2-b5b9-4962-8560-e3271028ec05 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:36.946488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 62/127] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:29:36 2022] PUT /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4/users/3eb5ec627dc1453ca7e211a5b47289af/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:29:36.947297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dcab2-b5b9-4962-8560-e3271028ec05 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:36.972972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd8d37-6634-449d-bdf1-bace8b20d120 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:36.979939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:37.031591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45aacc-1e1b-4247-89e6-f5f55ae46dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:37.034115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45aacc-1e1b-4247-89e6-f5f55ae46dbd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:37.034950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45aacc-1e1b-4247-89e6-f5f55ae46dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:37.035702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45aacc-1e1b-4247-89e6-f5f55ae46dbd None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:37.038659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45aacc-1e1b-4247-89e6-f5f55ae46dbd None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:37.041242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45aacc-1e1b-4247-89e6-f5f55ae46dbd None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:37.043771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 66/128] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:36 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:37.044869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45aacc-1e1b-4247-89e6-f5f55ae46dbd None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:37.090975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:37.157272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dcab2-b5b9-4962-8560-e3271028ec05 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:37.202246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45aacc-1e1b-4247-89e6-f5f55ae46dbd None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:37.202551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2646de3-7668-4aea-8d92-4ed5f40c566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:37.203441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2646de3-7668-4aea-8d92-4ed5f40c566f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:37.203900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2646de3-7668-4aea-8d92-4ed5f40c566f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:37.204271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2646de3-7668-4aea-8d92-4ed5f40c566f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:37.219587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2646de3-7668-4aea-8d92-4ed5f40c566f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-d2646de3-7668-4aea-8d92-4ed5f40c566f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:37.228429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:37.229334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:37.230132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 63/129] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:37.231147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2646de3-7668-4aea-8d92-4ed5f40c566f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:37.251865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c21db-c32d-4763-aa9f-b738ab4390d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:37.252703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c21db-c32d-4763-aa9f-b738ab4390d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:37.253069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c21db-c32d-4763-aa9f-b738ab4390d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:37.253495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c21db-c32d-4763-aa9f-b738ab4390d2 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4/users/c44d1cb246ea41a89355002d9d2481af/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:37.256139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c21db-c32d-4763-aa9f-b738ab4390d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=86e412e829cf47228db2532cb4c7c4e4, user_id=c44d1cb246ea41a89355002d9d2481af, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:37.257700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c21db-c32d-4763-aa9f-b738ab4390d2 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:37.328666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:37.369890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 67/130] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:29:37 2022] PUT /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4/users/c44d1cb246ea41a89355002d9d2481af/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:29:37.370293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2646de3-7668-4aea-8d92-4ed5f40c566f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:37.372009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c21db-c32d-4763-aa9f-b738ab4390d2 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:37.433085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f2e6e-3428-45e2-b524-bcd60becb433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:37.434009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2e6e-3428-45e2-b524-bcd60becb433 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:37.434479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2e6e-3428-45e2-b524-bcd60becb433 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:37.434942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2e6e-3428-45e2-b524-bcd60becb433 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:37.436374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f2e6e-3428-45e2-b524-bcd60becb433 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:37.438934 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f2e6e-3428-45e2-b524-bcd60becb433 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:37.441015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 64/131] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:37 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:37.442789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288f3ce-a3b8-4aaf-a7e1-ac69ca564465 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:37.443346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288f3ce-a3b8-4aaf-a7e1-ac69ca564465 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:37.443843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288f3ce-a3b8-4aaf-a7e1-ac69ca564465 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:37.445564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 65/132] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:37.447186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288f3ce-a3b8-4aaf-a7e1-ac69ca564465 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:37.521512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:37.535162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c21db-c32d-4763-aa9f-b738ab4390d2 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:37.578034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6836ae-6c29-4ddc-a707-d35ee963f130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:37.579453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6836ae-6c29-4ddc-a707-d35ee963f130 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:37.579734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6836ae-6c29-4ddc-a707-d35ee963f130 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:37.580008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6836ae-6c29-4ddc-a707-d35ee963f130 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4/users/3eb5ec627dc1453ca7e211a5b47289af/roles/e5de7bbccf114283851bcc0028d7cc51` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:37.583155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6836ae-6c29-4ddc-a707-d35ee963f130 None admin] RBAC: Authorizing `identity:create_grant(project_id=86e412e829cf47228db2532cb4c7c4e4, user_id=3eb5ec627dc1453ca7e211a5b47289af, role_id=e5de7bbccf114283851bcc0028d7cc51)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:37.584646 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6836ae-6c29-4ddc-a707-d35ee963f130 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:37.621653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288f3ce-a3b8-4aaf-a7e1-ac69ca564465 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:37.671829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667b057-af95-47f0-b049-b2586fb0d94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:37.672994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b057-af95-47f0-b049-b2586fb0d94d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:37.673350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b057-af95-47f0-b049-b2586fb0d94d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:37.673657 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667b057-af95-47f0-b049-b2586fb0d94d None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:37.674168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667b057-af95-47f0-b049-b2586fb0d94d None admin] RBAC: Adding query filter params (name=member) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:37.674459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667b057-af95-47f0-b049-b2586fb0d94d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:37.676978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667b057-af95-47f0-b049-b2586fb0d94d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:37.690787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 66/133] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:37.690787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27637e28-3aba-44e8-a1e2-b17481e06355 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:37.690787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27637e28-3aba-44e8-a1e2-b17481e06355 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:37.690787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27637e28-3aba-44e8-a1e2-b17481e06355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:37.696760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:37.714740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 68/134] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:29:37 2022] PUT /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4/users/3eb5ec627dc1453ca7e211a5b47289af/roles/e5de7bbccf114283851bcc0028d7cc51 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:29:37.715509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-27637e28-3aba-44e8-a1e2-b17481e06355 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:37.716902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-27637e28-3aba-44e8-a1e2-b17481e06355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:37.717794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6836ae-6c29-4ddc-a707-d35ee963f130 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:37.861870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 67/135] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:37.864860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27637e28-3aba-44e8-a1e2-b17481e06355 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:37.928649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6836ae-6c29-4ddc-a707-d35ee963f130 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:37.983160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c608c-3242-4460-8878-05666871fc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:37.984387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c608c-3242-4460-8878-05666871fc42 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:37.984875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c608c-3242-4460-8878-05666871fc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:37.985359 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c608c-3242-4460-8878-05666871fc42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:37.986311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c608c-3242-4460-8878-05666871fc42 None admin] RBAC: Adding query filter params (role.id=d8a54178fbc44a49ad0df8d85a2adf0b, scope.project.id=86e412e829cf47228db2532cb4c7c4e4, user.id=9565cb05f1204af6af9e9f4780d8cf0b) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:37.987017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c608c-3242-4460-8878-05666871fc42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:37.988938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c608c-3242-4460-8878-05666871fc42 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:37.995399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 69/136] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:37 2022] GET /identity/v3/role_assignments?user.id=9565cb05f1204af6af9e9f4780d8cf0b&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=d8a54178fbc44a49ad0df8d85a2adf0b => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:38.070914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27637e28-3aba-44e8-a1e2-b17481e06355 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:38.112214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:38.117529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8498c7-435d-4670-8942-0417582c7532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:38.118200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8498c7-435d-4670-8942-0417582c7532 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:38.118489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8498c7-435d-4670-8942-0417582c7532 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:38.118831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8498c7-435d-4670-8942-0417582c7532 None admin] PATH_INFO: `/v3/users/3eb5ec627dc1453ca7e211a5b47289af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:38.120040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8498c7-435d-4670-8942-0417582c7532 None admin] RBAC: Authorizing `identity:get_user(user_id=3eb5ec627dc1453ca7e211a5b47289af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:38.121665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8498c7-435d-4670-8942-0417582c7532 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:38.123531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 68/137] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:37 2022] GET /identity/v3/users/3eb5ec627dc1453ca7e211a5b47289af => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:38.129585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c608c-3242-4460-8878-05666871fc42 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:38.162267 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:38.286433 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c608c-3242-4460-8878-05666871fc42 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:38.337871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1258c-4a90-4fdb-84a5-86a1a9c28dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:38.339456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1258c-4a90-4fdb-84a5-86a1a9c28dba None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:38.340239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1258c-4a90-4fdb-84a5-86a1a9c28dba None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:38.340961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1258c-4a90-4fdb-84a5-86a1a9c28dba None admin] PATH_INFO: `/v3/projects/398402efcd674cf48c4694271975d2b1` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:38.343017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1258c-4a90-4fdb-84a5-86a1a9c28dba None admin] RBAC: Authorizing `identity:get_project(project_id=398402efcd674cf48c4694271975d2b1)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:38.345988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1258c-4a90-4fdb-84a5-86a1a9c28dba None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:38.348741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 70/138] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:38 2022] GET /identity/v3/projects/398402efcd674cf48c4694271975d2b1 => generated 312 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:38.356255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8498c7-435d-4670-8942-0417582c7532 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:38.505458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8498c7-435d-4670-8942-0417582c7532 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:38.519389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1258c-4a90-4fdb-84a5-86a1a9c28dba None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:38.635564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db109f9c-be3b-4690-96f7-08040629af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:38.636658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db109f9c-be3b-4690-96f7-08040629af62 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:38.637146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db109f9c-be3b-4690-96f7-08040629af62 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:38.637625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db109f9c-be3b-4690-96f7-08040629af62 None admin] PATH_INFO: `/v3/projects/398402efcd674cf48c4694271975d2b1/users/3eb5ec627dc1453ca7e211a5b47289af/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:38.644178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db109f9c-be3b-4690-96f7-08040629af62 None admin] RBAC: Authorizing `identity:create_grant(project_id=398402efcd674cf48c4694271975d2b1, user_id=3eb5ec627dc1453ca7e211a5b47289af, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:38.646039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db109f9c-be3b-4690-96f7-08040629af62 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:38.764752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:38.802058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 69/139] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:29:38 2022] PUT /identity/v3/projects/398402efcd674cf48c4694271975d2b1/users/3eb5ec627dc1453ca7e211a5b47289af/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:29:38.808301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1258c-4a90-4fdb-84a5-86a1a9c28dba None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:38.892387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627a06-07a5-42a2-9b81-86e80acba3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:38.893351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627a06-07a5-42a2-9b81-86e80acba3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:38.893639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627a06-07a5-42a2-9b81-86e80acba3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:38.893921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627a06-07a5-42a2-9b81-86e80acba3c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:38.895477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b627a06-07a5-42a2-9b81-86e80acba3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:38.897353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b627a06-07a5-42a2-9b81-86e80acba3c6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:38.899402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 71/140] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:38.906109 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db109f9c-be3b-4690-96f7-08040629af62 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:39.074311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:39.154865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db109f9c-be3b-4690-96f7-08040629af62 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:39.224260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb21a87-fcbf-4e72-86bd-e604e456e401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:39.224901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb21a87-fcbf-4e72-86bd-e604e456e401 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:39.225195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb21a87-fcbf-4e72-86bd-e604e456e401 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:39.225558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb21a87-fcbf-4e72-86bd-e604e456e401 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:39.226256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb21a87-fcbf-4e72-86bd-e604e456e401 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:39.227847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb21a87-fcbf-4e72-86bd-e604e456e401 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:39.397819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 70/141] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:29:38 2022] POST /identity/v3/users => generated 283 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:40.118416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49798f93-8c38-4b9a-b698-05171b329c42 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:40.119091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49798f93-8c38-4b9a-b698-05171b329c42 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:40.119531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49798f93-8c38-4b9a-b698-05171b329c42 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:40.122104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 72/142] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:40.139479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ff373-469e-4eb8-a73a-b56388636d39 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:40.140005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ff373-469e-4eb8-a73a-b56388636d39 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:40.140547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ff373-469e-4eb8-a73a-b56388636d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:40.182611 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f22ff373-469e-4eb8-a73a-b56388636d39 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:40.184743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f22ff373-469e-4eb8-a73a-b56388636d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:40.296219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 71/143] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:40.339009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4639-55f9-434b-8aca-941bb9e38c21 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:40.339517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4639-55f9-434b-8aca-941bb9e38c21 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:40.340013 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4639-55f9-434b-8aca-941bb9e38c21 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:40.342524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 73/144] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:40.350318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ed8f-9b34-46c1-9429-782fe9dd43d3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:40.350853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ed8f-9b34-46c1-9429-782fe9dd43d3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:40.351344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ed8f-9b34-46c1-9429-782fe9dd43d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:40.396480 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-dfa5ed8f-9b34-46c1-9429-782fe9dd43d3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:40.398556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dfa5ed8f-9b34-46c1-9429-782fe9dd43d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:40.494023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:40.544202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 72/145] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:41.072568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4639-55f9-434b-8aca-941bb9e38c21 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:41.283754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4639-55f9-434b-8aca-941bb9e38c21 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:41.346768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea864ea-411d-45f8-b1a0-5fd72ee30e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:41.347893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea864ea-411d-45f8-b1a0-5fd72ee30e86 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:41.348386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea864ea-411d-45f8-b1a0-5fd72ee30e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:41.348876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea864ea-411d-45f8-b1a0-5fd72ee30e86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:41.368461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea864ea-411d-45f8-b1a0-5fd72ee30e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-4ea864ea-411d-45f8-b1a0-5fd72ee30e86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:41.378543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:41.380104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:41.382378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 74/146] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:41.389114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5ed8f-9b34-46c1-9429-782fe9dd43d3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:41.566119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5ed8f-9b34-46c1-9429-782fe9dd43d3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:41.576884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea864ea-411d-45f8-b1a0-5fd72ee30e86 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:41.611923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fee0ac-3e32-419d-aa72-b1607f5bf518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:41.612552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fee0ac-3e32-419d-aa72-b1607f5bf518 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:41.612840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fee0ac-3e32-419d-aa72-b1607f5bf518 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:41.613121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fee0ac-3e32-419d-aa72-b1607f5bf518 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:41.613816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fee0ac-3e32-419d-aa72-b1607f5bf518 None admin] RBAC: Adding query filter params (name=member) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:41.614124 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fee0ac-3e32-419d-aa72-b1607f5bf518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:41.614906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:41.616086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fee0ac-3e32-419d-aa72-b1607f5bf518 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:41.627411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 73/147] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:41.636062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fee0ac-3e32-419d-aa72-b1607f5bf518 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:41.686256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:41.764689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fee0ac-3e32-419d-aa72-b1607f5bf518 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:41.805525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b22362-8673-4fea-a3c2-02b42468eb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:41.806229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22362-8673-4fea-a3c2-02b42468eb07 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:41.806626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22362-8673-4fea-a3c2-02b42468eb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:41.807118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b22362-8673-4fea-a3c2-02b42468eb07 None admin] PATH_INFO: `/v3/users/3eb5ec627dc1453ca7e211a5b47289af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:41.808339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b22362-8673-4fea-a3c2-02b42468eb07 None admin] RBAC: Authorizing `identity:get_user(user_id=3eb5ec627dc1453ca7e211a5b47289af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:41.809889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b22362-8673-4fea-a3c2-02b42468eb07 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:41.811544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 74/148] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:41 2022] GET /identity/v3/users/3eb5ec627dc1453ca7e211a5b47289af => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:41.820652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b22362-8673-4fea-a3c2-02b42468eb07 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:41.876489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea864ea-411d-45f8-b1a0-5fd72ee30e86 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:41.966692 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55b50c-6109-4c20-8f58-e872c32db7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:41.967758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55b50c-6109-4c20-8f58-e872c32db7fd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:41.968249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55b50c-6109-4c20-8f58-e872c32db7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:41.969287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55b50c-6109-4c20-8f58-e872c32db7fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:41.978058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b22362-8673-4fea-a3c2-02b42468eb07 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:41.988707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:42.031693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55b50c-6109-4c20-8f58-e872c32db7fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-fc55b50c-6109-4c20-8f58-e872c32db7fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:42.036708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:42.038281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:42.039867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 75/149] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:29:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:29:42.051450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a373f-dd73-4194-8b10-6814293be615 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:42.051806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a373f-dd73-4194-8b10-6814293be615 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:42.052035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a373f-dd73-4194-8b10-6814293be615 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:42.053586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 76/150] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:42 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:42.055414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a373f-dd73-4194-8b10-6814293be615 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:42.073108 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad8981-f528-49f5-b86c-ecfd6dad7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:42.074032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8981-f528-49f5-b86c-ecfd6dad7759 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:42.074534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8981-f528-49f5-b86c-ecfd6dad7759 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:42.075020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8981-f528-49f5-b86c-ecfd6dad7759 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:42.244153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad8981-f528-49f5-b86c-ecfd6dad7759 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:42.245655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad8981-f528-49f5-b86c-ecfd6dad7759 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:42.280898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:42.345659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a373f-dd73-4194-8b10-6814293be615 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:42.378058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 75/151] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:42.380170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d5847-480b-4ffc-825f-efb15e399950 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:42.380665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d5847-480b-4ffc-825f-efb15e399950 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:42.386534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d5847-480b-4ffc-825f-efb15e399950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:42.416300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:42.425618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-722d5847-480b-4ffc-825f-efb15e399950 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:42.427696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-722d5847-480b-4ffc-825f-efb15e399950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:42.455640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22654df-f5e9-4e9b-a08c-d84dd6940bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:42.457138 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22654df-f5e9-4e9b-a08c-d84dd6940bbf None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:42.457887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22654df-f5e9-4e9b-a08c-d84dd6940bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:42.458637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22654df-f5e9-4e9b-a08c-d84dd6940bbf None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:42.461450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22654df-f5e9-4e9b-a08c-d84dd6940bbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:42.462291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22654df-f5e9-4e9b-a08c-d84dd6940bbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:42.465249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22654df-f5e9-4e9b-a08c-d84dd6940bbf None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:42.501816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 77/152] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:42.503795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22654df-f5e9-4e9b-a08c-d84dd6940bbf None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:42.686239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 76/153] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:42.689641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d5847-480b-4ffc-825f-efb15e399950 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:42.887488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d5847-480b-4ffc-825f-efb15e399950 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:42.904980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22654df-f5e9-4e9b-a08c-d84dd6940bbf None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:42.957346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264fd6a-9a55-4698-b7c0-04527d36a32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:42.957900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264fd6a-9a55-4698-b7c0-04527d36a32f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:42.958217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264fd6a-9a55-4698-b7c0-04527d36a32f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:42.958479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264fd6a-9a55-4698-b7c0-04527d36a32f None admin] PATH_INFO: `/v3/users/c44d1cb246ea41a89355002d9d2481af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:42.963221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264fd6a-9a55-4698-b7c0-04527d36a32f None admin] RBAC: Authorizing `identity:get_user(user_id=c44d1cb246ea41a89355002d9d2481af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:42.964713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264fd6a-9a55-4698-b7c0-04527d36a32f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:42.966415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 77/154] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:42 2022] GET /identity/v3/users/c44d1cb246ea41a89355002d9d2481af => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:42.967595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ecbd6-74c5-473e-8aba-d8cab4c56fc9 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:42.967869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ecbd6-74c5-473e-8aba-d8cab4c56fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:42.968134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ecbd6-74c5-473e-8aba-d8cab4c56fc9 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:42.970286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 78/155] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:42.973906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e2d1e-8762-43a7-80c4-f10107a3d351 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:42.973906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e2d1e-8762-43a7-80c4-f10107a3d351 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:42.973906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e2d1e-8762-43a7-80c4-f10107a3d351 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:42.973906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 79/156] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:42.978852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e2d1e-8762-43a7-80c4-f10107a3d351 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:42.986329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:43.034971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd94a8a-9cf8-498d-8ccd-2661c9b06d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:43.035790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd94a8a-9cf8-498d-8ccd-2661c9b06d35 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:43.036076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd94a8a-9cf8-498d-8ccd-2661c9b06d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:43.036353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd94a8a-9cf8-498d-8ccd-2661c9b06d35 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:43.038329 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd94a8a-9cf8-498d-8ccd-2661c9b06d35 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:43.039098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd94a8a-9cf8-498d-8ccd-2661c9b06d35 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:43.040749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 78/157] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:42 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 538 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:43.042194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8d69c-6397-481d-8487-1c02adc611ca None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:43.042496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8d69c-6397-481d-8487-1c02adc611ca None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:43.042775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8d69c-6397-481d-8487-1c02adc611ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:43.064440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:43.085355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-edd8d69c-6397-481d-8487-1c02adc611ca None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:43.086590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-edd8d69c-6397-481d-8487-1c02adc611ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:43.127498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e2d1e-8762-43a7-80c4-f10107a3d351 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:43.173023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d8ede-0e10-4d89-94ee-02ca2d201dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:43.173642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d8ede-0e10-4d89-94ee-02ca2d201dbd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:43.173921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d8ede-0e10-4d89-94ee-02ca2d201dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:43.174190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d8ede-0e10-4d89-94ee-02ca2d201dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:43.254542 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 79/158] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:43.255603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924b9e5-6f2b-4cb7-b6ec-fa0db10b3582 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:43.255887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924b9e5-6f2b-4cb7-b6ec-fa0db10b3582 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:43.256162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924b9e5-6f2b-4cb7-b6ec-fa0db10b3582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:43.311974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5924b9e5-6f2b-4cb7-b6ec-fa0db10b3582 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:43.314291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5924b9e5-6f2b-4cb7-b6ec-fa0db10b3582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:43.375411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d8ede-0e10-4d89-94ee-02ca2d201dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:43.376927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d8ede-0e10-4d89-94ee-02ca2d201dbd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:43.379886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:43.455260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 80/159] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:43.456565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924b9e5-6f2b-4cb7-b6ec-fa0db10b3582 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:43.537147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:43.585430 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924b9e5-6f2b-4cb7-b6ec-fa0db10b3582 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:43.630566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77887a6-89d0-4837-be4c-0610965d8eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:43.631456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77887a6-89d0-4837-be4c-0610965d8eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:43.631744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77887a6-89d0-4837-be4c-0610965d8eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:43.632019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77887a6-89d0-4837-be4c-0610965d8eb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:43.632769 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77887a6-89d0-4837-be4c-0610965d8eb6 None admin] RBAC: Adding query filter params (role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=86e412e829cf47228db2532cb4c7c4e4, user.id=3eb5ec627dc1453ca7e211a5b47289af) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:43.633081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77887a6-89d0-4837-be4c-0610965d8eb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:43.635249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77887a6-89d0-4837-be4c-0610965d8eb6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:43.642900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 81/160] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:43 2022] GET /identity/v3/role_assignments?user.id=3eb5ec627dc1453ca7e211a5b47289af&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:43.644068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77887a6-89d0-4837-be4c-0610965d8eb6 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:43.678460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 80/161] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:29:43.706720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d8ede-0e10-4d89-94ee-02ca2d201dbd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:43.822387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77887a6-89d0-4837-be4c-0610965d8eb6 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:43.861542 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:43.880465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cb007-e4c4-45f0-93e3-66222502fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:43.881404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cb007-e4c4-45f0-93e3-66222502fc73 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:43.881837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cb007-e4c4-45f0-93e3-66222502fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:43.882270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cb007-e4c4-45f0-93e3-66222502fc73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:43.912055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cb007-e4c4-45f0-93e3-66222502fc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-612cb007-e4c4-45f0-93e3-66222502fc73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:43.927486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:43.928828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:43.929858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 82/162] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:29:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:43.944308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cb007-e4c4-45f0-93e3-66222502fc73 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:43.996912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d8ede-0e10-4d89-94ee-02ca2d201dbd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:44.003577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:44.119446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23326c4f-ef43-425b-b984-ab9e6723ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:44.120038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23326c4f-ef43-425b-b984-ab9e6723ba25 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:44.120325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23326c4f-ef43-425b-b984-ab9e6723ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:44.120611 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23326c4f-ef43-425b-b984-ab9e6723ba25 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:44.122631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23326c4f-ef43-425b-b984-ab9e6723ba25 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:44.124555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23326c4f-ef43-425b-b984-ab9e6723ba25 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:44.128207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 81/163] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:43 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:44.129137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23326c4f-ef43-425b-b984-ab9e6723ba25 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:44.306808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cb007-e4c4-45f0-93e3-66222502fc73 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:44.369975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23326c4f-ef43-425b-b984-ab9e6723ba25 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:44.388122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3755fd-d00d-4faf-a9c2-ce7793a83ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:44.389181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3755fd-d00d-4faf-a9c2-ce7793a83ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:44.389665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3755fd-d00d-4faf-a9c2-ce7793a83ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:44.390137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3755fd-d00d-4faf-a9c2-ce7793a83ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:44.392293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3755fd-d00d-4faf-a9c2-ce7793a83ed2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:44.392822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3755fd-d00d-4faf-a9c2-ce7793a83ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:44.394462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3755fd-d00d-4faf-a9c2-ce7793a83ed2 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:44.409828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fda92-5ca3-40bc-9194-413c2421aae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:44.410479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fda92-5ca3-40bc-9194-413c2421aae2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:44.410789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fda92-5ca3-40bc-9194-413c2421aae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:44.411119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fda92-5ca3-40bc-9194-413c2421aae2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:44.414126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 83/164] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:29:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:44.415114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3755fd-d00d-4faf-a9c2-ce7793a83ed2 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:44.423069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fda92-5ca3-40bc-9194-413c2421aae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-624fda92-5ca3-40bc-9194-413c2421aae2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:44.428459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:44.429871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:44.430758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 82/165] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:44.431432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fda92-5ca3-40bc-9194-413c2421aae2 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:44.473933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:44.525345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:44.603172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3755fd-d00d-4faf-a9c2-ce7793a83ed2 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:44.610240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fda92-5ca3-40bc-9194-413c2421aae2 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:44.680621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decc2af-4d51-480d-8f0a-9a44e3b4bec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:44.681680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decc2af-4d51-480d-8f0a-9a44e3b4bec1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:44.682166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decc2af-4d51-480d-8f0a-9a44e3b4bec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:44.682668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decc2af-4d51-480d-8f0a-9a44e3b4bec1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:44.691255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc29ee0-bd9b-4ad6-9db0-f0b31114b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:44.691898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc29ee0-bd9b-4ad6-9db0-f0b31114b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:44.692185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc29ee0-bd9b-4ad6-9db0-f0b31114b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:44.692467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc29ee0-bd9b-4ad6-9db0-f0b31114b7d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:44.693217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc29ee0-bd9b-4ad6-9db0-f0b31114b7d0 None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=86e412e829cf47228db2532cb4c7c4e4, user.id=c44d1cb246ea41a89355002d9d2481af) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:44.693532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc29ee0-bd9b-4ad6-9db0-f0b31114b7d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:44.695029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc29ee0-bd9b-4ad6-9db0-f0b31114b7d0 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:44.703466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8decc2af-4d51-480d-8f0a-9a44e3b4bec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-8decc2af-4d51-480d-8f0a-9a44e3b4bec1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:44.713330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:44.715811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:44.717864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:44.720654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 84/166] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:44 2022] GET /identity/v3/role_assignments?user.id=c44d1cb246ea41a89355002d9d2481af&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=9e904216086e4a06abf9216b0c863f89 => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:44.720654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc29ee0-bd9b-4ad6-9db0-f0b31114b7d0 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:44.720924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 83/167] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:44.720924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decc2af-4d51-480d-8f0a-9a44e3b4bec1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:44.912803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc29ee0-bd9b-4ad6-9db0-f0b31114b7d0 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:44.923491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:44.941568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decc2af-4d51-480d-8f0a-9a44e3b4bec1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:44.945977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:45.000793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71182cf4-181d-4369-ab33-a936b3193ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:45.001661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71182cf4-181d-4369-ab33-a936b3193ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.001947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71182cf4-181d-4369-ab33-a936b3193ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.002224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71182cf4-181d-4369-ab33-a936b3193ad5 None admin] PATH_INFO: `/v3/users/3eb5ec627dc1453ca7e211a5b47289af` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.005274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71182cf4-181d-4369-ab33-a936b3193ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=3eb5ec627dc1453ca7e211a5b47289af)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:45.006803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71182cf4-181d-4369-ab33-a936b3193ad5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:45.008707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 85/168] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:44 2022] GET /identity/v3/users/3eb5ec627dc1453ca7e211a5b47289af => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:45.014124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71182cf4-181d-4369-ab33-a936b3193ad5 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:45.030618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17820bb-8eef-48a2-a212-04cb4e9a9eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:45.032450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17820bb-8eef-48a2-a212-04cb4e9a9eca None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.032450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17820bb-8eef-48a2-a212-04cb4e9a9eca None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.032450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17820bb-8eef-48a2-a212-04cb4e9a9eca None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.033070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17820bb-8eef-48a2-a212-04cb4e9a9eca None admin] RBAC: Adding query filter params (name=reader) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:45.033379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17820bb-8eef-48a2-a212-04cb4e9a9eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:45.035374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17820bb-8eef-48a2-a212-04cb4e9a9eca None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:45.060492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 84/169] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:45.061801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ab441-6da6-48df-b1a6-5d48336202b3 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.062100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ab441-6da6-48df-b1a6-5d48336202b3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.062990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ab441-6da6-48df-b1a6-5d48336202b3 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.064770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 85/170] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:45.065154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ab441-6da6-48df-b1a6-5d48336202b3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:45.194358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71182cf4-181d-4369-ab33-a936b3193ad5 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:45.233360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ab441-6da6-48df-b1a6-5d48336202b3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:45.238934 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef568b-88b7-4843-9358-27e7cfe1151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:45.239915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef568b-88b7-4843-9358-27e7cfe1151b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.240436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef568b-88b7-4843-9358-27e7cfe1151b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.240914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef568b-88b7-4843-9358-27e7cfe1151b None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.242020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef568b-88b7-4843-9358-27e7cfe1151b None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:45.242550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef568b-88b7-4843-9358-27e7cfe1151b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:45.244162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef568b-88b7-4843-9358-27e7cfe1151b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:45.255441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 86/171] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:45.256680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef568b-88b7-4843-9358-27e7cfe1151b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:45.302852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d20ca-3bfc-44c2-abab-777188312fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:45.303921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d20ca-3bfc-44c2-abab-777188312fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.304418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d20ca-3bfc-44c2-abab-777188312fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.304912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d20ca-3bfc-44c2-abab-777188312fc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.310626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:45.351389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:45.386039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef568b-88b7-4843-9358-27e7cfe1151b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:45.427835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa135a6c-d5d3-4c8e-9796-6e3b91357a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:45.428482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa135a6c-d5d3-4c8e-9796-6e3b91357a4c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.428763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa135a6c-d5d3-4c8e-9796-6e3b91357a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.429038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa135a6c-d5d3-4c8e-9796-6e3b91357a4c None admin] PATH_INFO: `/v3/users/9565cb05f1204af6af9e9f4780d8cf0b` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.430205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa135a6c-d5d3-4c8e-9796-6e3b91357a4c None admin] RBAC: Authorizing `identity:get_user(user_id=9565cb05f1204af6af9e9f4780d8cf0b)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:45.431821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d20ca-3bfc-44c2-abab-777188312fc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:45.432094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa135a6c-d5d3-4c8e-9796-6e3b91357a4c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:45.433295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d20ca-3bfc-44c2-abab-777188312fc4 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:45.434499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 87/172] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:45 2022] GET /identity/v3/users/9565cb05f1204af6af9e9f4780d8cf0b => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:45.434499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738afe49-aeae-4cce-adff-a70abf5d44da None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.436132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738afe49-aeae-4cce-adff-a70abf5d44da None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.436132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738afe49-aeae-4cce-adff-a70abf5d44da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.455563 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-738afe49-aeae-4cce-adff-a70abf5d44da None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:45.456697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-738afe49-aeae-4cce-adff-a70abf5d44da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:45.548069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 88/173] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:45.549421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738afe49-aeae-4cce-adff-a70abf5d44da None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:45.575969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 86/174] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:45.577093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c1cf3-a491-4fe8-9f92-d08fc0b9d53c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.577387 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c1cf3-a491-4fe8-9f92-d08fc0b9d53c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.577672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c1cf3-a491-4fe8-9f92-d08fc0b9d53c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.579349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 87/175] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:45.584720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741c387-50bb-4196-ad38-05c295a26b6c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.585245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741c387-50bb-4196-ad38-05c295a26b6c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.585696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741c387-50bb-4196-ad38-05c295a26b6c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.588138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 88/176] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:45.589410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741c387-50bb-4196-ad38-05c295a26b6c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:45.608225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:45.675582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738afe49-aeae-4cce-adff-a70abf5d44da None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:45.690716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:45.727301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49fd41-deeb-45fd-93f5-3abf768618be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:45.727867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49fd41-deeb-45fd-93f5-3abf768618be None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.728150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49fd41-deeb-45fd-93f5-3abf768618be None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.728424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49fd41-deeb-45fd-93f5-3abf768618be None admin] PATH_INFO: `/v3/users/3eb5ec627dc1453ca7e211a5b47289af` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.729586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49fd41-deeb-45fd-93f5-3abf768618be None admin] RBAC: Authorizing `identity:get_user(user_id=3eb5ec627dc1453ca7e211a5b47289af)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:45.731117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49fd41-deeb-45fd-93f5-3abf768618be None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:45.733177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 89/177] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:45 2022] GET /identity/v3/users/3eb5ec627dc1453ca7e211a5b47289af => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:45.738570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49fd41-deeb-45fd-93f5-3abf768618be None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:45.774306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741c387-50bb-4196-ad38-05c295a26b6c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:45.819421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ccf13-a186-4d2f-83eb-b62f408e5431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:45.820767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ccf13-a186-4d2f-83eb-b62f408e5431 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.820767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ccf13-a186-4d2f-83eb-b62f408e5431 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.821888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ccf13-a186-4d2f-83eb-b62f408e5431 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.823278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ccf13-a186-4d2f-83eb-b62f408e5431 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:45.825040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ccf13-a186-4d2f-83eb-b62f408e5431 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:45.827067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 89/178] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:45 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:45.833051 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a552f-1750-4d69-b36f-5e58a7076ed5 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.833329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a552f-1750-4d69-b36f-5e58a7076ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.833640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a552f-1750-4d69-b36f-5e58a7076ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.854456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-640a552f-1750-4d69-b36f-5e58a7076ed5 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:45.855767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-640a552f-1750-4d69-b36f-5e58a7076ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:45.886436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49fd41-deeb-45fd-93f5-3abf768618be None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:45.902978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:45.934943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5c2f5-4085-43f3-80e1-38201abe6353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:45.935824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5c2f5-4085-43f3-80e1-38201abe6353 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.936386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5c2f5-4085-43f3-80e1-38201abe6353 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.936756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5c2f5-4085-43f3-80e1-38201abe6353 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.938635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5c2f5-4085-43f3-80e1-38201abe6353 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:45.941084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5c2f5-4085-43f3-80e1-38201abe6353 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:45.941325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 90/179] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:45.942689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9455340-ef81-4d1d-b892-3abbb7d4bc4d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:45.942975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9455340-ef81-4d1d-b892-3abbb7d4bc4d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:45.943246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9455340-ef81-4d1d-b892-3abbb7d4bc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:45.950889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 90/180] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:45 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:45.950889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5c2f5-4085-43f3-80e1-38201abe6353 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:45.969973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b9455340-ef81-4d1d-b892-3abbb7d4bc4d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:45.971448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b9455340-ef81-4d1d-b892-3abbb7d4bc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:46.043394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:46.078541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 91/181] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:46.078541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9455340-ef81-4d1d-b892-3abbb7d4bc4d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:46.127635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5c2f5-4085-43f3-80e1-38201abe6353 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:46.176944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507eee8-a1d5-47b6-8c6a-d6799537b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:46.177819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507eee8-a1d5-47b6-8c6a-d6799537b68e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:46.178098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507eee8-a1d5-47b6-8c6a-d6799537b68e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:46.178370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507eee8-a1d5-47b6-8c6a-d6799537b68e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:46.201502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9455340-ef81-4d1d-b892-3abbb7d4bc4d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:46.239205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:46.240515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb7bfe-2876-4ac2-89cb-7dd1273155a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:46.241190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb7bfe-2876-4ac2-89cb-7dd1273155a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:46.241471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb7bfe-2876-4ac2-89cb-7dd1273155a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:46.241749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb7bfe-2876-4ac2-89cb-7dd1273155a3 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4/users/9565cb05f1204af6af9e9f4780d8cf0b/roles/d8a54178fbc44a49ad0df8d85a2adf0b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:46.243687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb7bfe-2876-4ac2-89cb-7dd1273155a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=86e412e829cf47228db2532cb4c7c4e4, user_id=9565cb05f1204af6af9e9f4780d8cf0b, role_id=d8a54178fbc44a49ad0df8d85a2adf0b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:46.245153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb7bfe-2876-4ac2-89cb-7dd1273155a3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:46.263954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507eee8-a1d5-47b6-8c6a-d6799537b68e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:46.265502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507eee8-a1d5-47b6-8c6a-d6799537b68e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:46.323403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 92/182] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:29:46 2022] PUT /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4/users/9565cb05f1204af6af9e9f4780d8cf0b/roles/d8a54178fbc44a49ad0df8d85a2adf0b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:29:46.324083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb7bfe-2876-4ac2-89cb-7dd1273155a3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:46.387740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:46.393164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 91/183] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:46.394233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507eee8-a1d5-47b6-8c6a-d6799537b68e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:46.483307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb7bfe-2876-4ac2-89cb-7dd1273155a3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:46.522760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:46.527381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e3f5a-5eb2-433e-89af-263c32f9459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:46.528230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e3f5a-5eb2-433e-89af-263c32f9459e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:46.528660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e3f5a-5eb2-433e-89af-263c32f9459e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:46.529102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e3f5a-5eb2-433e-89af-263c32f9459e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:46.546409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e3f5a-5eb2-433e-89af-263c32f9459e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-c50e3f5a-5eb2-433e-89af-263c32f9459e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:46.552518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:46.553536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:46.554519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:46.556787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 93/184] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:29:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:46.557780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e3f5a-5eb2-433e-89af-263c32f9459e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:46.565373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507eee8-a1d5-47b6-8c6a-d6799537b68e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:46.618665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a515c-0e58-42d4-972f-3f9556a4b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:46.619268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a515c-0e58-42d4-972f-3f9556a4b471 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:46.619549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a515c-0e58-42d4-972f-3f9556a4b471 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:46.619822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a515c-0e58-42d4-972f-3f9556a4b471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:46.620570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a515c-0e58-42d4-972f-3f9556a4b471 None admin] RBAC: Adding query filter params (role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=86e412e829cf47228db2532cb4c7c4e4, user.id=3eb5ec627dc1453ca7e211a5b47289af) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:46.620879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a515c-0e58-42d4-972f-3f9556a4b471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:46.622358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a515c-0e58-42d4-972f-3f9556a4b471 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:46.629960 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 92/185] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:46 2022] GET /identity/v3/role_assignments?user.id=3eb5ec627dc1453ca7e211a5b47289af&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:46.630818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a515c-0e58-42d4-972f-3f9556a4b471 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:46.738537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e3f5a-5eb2-433e-89af-263c32f9459e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:46.741668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:46.798913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d3efe-f5ac-42ef-85b7-b22826e3aeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:46.800775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d3efe-f5ac-42ef-85b7-b22826e3aeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:46.801451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d3efe-f5ac-42ef-85b7-b22826e3aeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:46.801936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d3efe-f5ac-42ef-85b7-b22826e3aeb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:46.804962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d3efe-f5ac-42ef-85b7-b22826e3aeb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:46.808507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d3efe-f5ac-42ef-85b7-b22826e3aeb5 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:46.809409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a515c-0e58-42d4-972f-3f9556a4b471 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:46.814263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 94/186] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:46.815768 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d3efe-f5ac-42ef-85b7-b22826e3aeb5 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:46.869575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d9e12-45d6-46ec-a7cb-2bfd5e2bd3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:46.870848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d9e12-45d6-46ec-a7cb-2bfd5e2bd3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:46.871142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d9e12-45d6-46ec-a7cb-2bfd5e2bd3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:46.871637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d9e12-45d6-46ec-a7cb-2bfd5e2bd3a1 None admin] PATH_INFO: `/v3/projects/398402efcd674cf48c4694271975d2b1` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:46.874995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d9e12-45d6-46ec-a7cb-2bfd5e2bd3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=398402efcd674cf48c4694271975d2b1)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:46.877583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d9e12-45d6-46ec-a7cb-2bfd5e2bd3a1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:46.881023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 93/187] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:46 2022] GET /identity/v3/projects/398402efcd674cf48c4694271975d2b1 => generated 312 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:46.882311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d9e12-45d6-46ec-a7cb-2bfd5e2bd3a1 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:46.900912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:47.011234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d3efe-f5ac-42ef-85b7-b22826e3aeb5 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:47.017642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d9e12-45d6-46ec-a7cb-2bfd5e2bd3a1 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:47.090327 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060786a-29a6-4d39-8cfb-77103cbe3f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:47.090671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fdf3f-3071-4c22-88e4-cc528ca544d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:47.091066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fdf3f-3071-4c22-88e4-cc528ca544d4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.091407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fdf3f-3071-4c22-88e4-cc528ca544d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.091730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fdf3f-3071-4c22-88e4-cc528ca544d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.092078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060786a-29a6-4d39-8cfb-77103cbe3f6a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.092531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fdf3f-3071-4c22-88e4-cc528ca544d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:47.092633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060786a-29a6-4d39-8cfb-77103cbe3f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.092875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fdf3f-3071-4c22-88e4-cc528ca544d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:47.093281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060786a-29a6-4d39-8cfb-77103cbe3f6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.094551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fdf3f-3071-4c22-88e4-cc528ca544d4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:47.106554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 94/188] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:47.106554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fdf3f-3071-4c22-88e4-cc528ca544d4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:47.110533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060786a-29a6-4d39-8cfb-77103cbe3f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-e060786a-29a6-4d39-8cfb-77103cbe3f6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:47.115504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:47.116576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 95/189] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:29:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:47.117448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060786a-29a6-4d39-8cfb-77103cbe3f6a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:47.164878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:47.196123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:47.230894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fdf3f-3071-4c22-88e4-cc528ca544d4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:47.270335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f3fc4-0d08-49f6-871b-942aac877241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:47.270993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f3fc4-0d08-49f6-871b-942aac877241 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.271276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f3fc4-0d08-49f6-871b-942aac877241 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.271556 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f3fc4-0d08-49f6-871b-942aac877241 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.272215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f3fc4-0d08-49f6-871b-942aac877241 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:47.273670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f3fc4-0d08-49f6-871b-942aac877241 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:47.277569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060786a-29a6-4d39-8cfb-77103cbe3f6a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:47.329180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b54af-7a1b-454a-bdc7-cace505fee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:47.329795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b54af-7a1b-454a-bdc7-cace505fee08 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.330129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b54af-7a1b-454a-bdc7-cace505fee08 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.330401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b54af-7a1b-454a-bdc7-cace505fee08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.331184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b54af-7a1b-454a-bdc7-cace505fee08 None admin] RBAC: Adding query filter params (role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=398402efcd674cf48c4694271975d2b1, user.id=3eb5ec627dc1453ca7e211a5b47289af) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:47.331608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b54af-7a1b-454a-bdc7-cace505fee08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:47.333092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b54af-7a1b-454a-bdc7-cace505fee08 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:47.339640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 96/190] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:47 2022] GET /identity/v3/role_assignments?user.id=3eb5ec627dc1453ca7e211a5b47289af&scope.project.id=398402efcd674cf48c4694271975d2b1&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:47.340441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b54af-7a1b-454a-bdc7-cace505fee08 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:47.390156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 95/191] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:29:47 2022] POST /identity/v3/users => generated 297 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:47.392731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f3fc4-0d08-49f6-871b-942aac877241 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:47.530658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f3fc4-0d08-49f6-871b-942aac877241 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:47.559952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:47.570841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd332b-4d43-4453-89f3-50b9beebe9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:47.571534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:47.572446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd332b-4d43-4453-89f3-50b9beebe9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.573079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd332b-4d43-4453-89f3-50b9beebe9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.573536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd332b-4d43-4453-89f3-50b9beebe9c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.574405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd332b-4d43-4453-89f3-50b9beebe9c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:47.574919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd332b-4d43-4453-89f3-50b9beebe9c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:47.576514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd332b-4d43-4453-89f3-50b9beebe9c1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:47.587615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 96/192] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:29:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:47.593142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b54af-7a1b-454a-bdc7-cace505fee08 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:47.598555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd332b-4d43-4453-89f3-50b9beebe9c1 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:47.661627 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab003b41-5b26-4454-9009-edf66060b817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:47.662421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab003b41-5b26-4454-9009-edf66060b817 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.663993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab003b41-5b26-4454-9009-edf66060b817 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.664366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab003b41-5b26-4454-9009-edf66060b817 None admin] PATH_INFO: `/v3/users/14b0836d896d4c0eb85eb9f8416a64ac` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.685534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab003b41-5b26-4454-9009-edf66060b817 None admin] RBAC: Authorizing `identity:get_user(user_id=14b0836d896d4c0eb85eb9f8416a64ac)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:47.687197 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab003b41-5b26-4454-9009-edf66060b817 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:47.689345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 97/193] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:47 2022] GET /identity/v3/users/14b0836d896d4c0eb85eb9f8416a64ac => generated 283 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:47.701343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab003b41-5b26-4454-9009-edf66060b817 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:47.757466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd332b-4d43-4453-89f3-50b9beebe9c1 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:47.822796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bed052-96b3-4bf0-8bba-a02b691c2832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:47.823537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bed052-96b3-4bf0-8bba-a02b691c2832 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.824800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bed052-96b3-4bf0-8bba-a02b691c2832 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.824800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bed052-96b3-4bf0-8bba-a02b691c2832 None admin] PATH_INFO: `/v3/users/14b0836d896d4c0eb85eb9f8416a64ac` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.826408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bed052-96b3-4bf0-8bba-a02b691c2832 None admin] RBAC: Authorizing `identity:get_user(user_id=14b0836d896d4c0eb85eb9f8416a64ac)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:47.827982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bed052-96b3-4bf0-8bba-a02b691c2832 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:47.830128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 97/194] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:47 2022] GET /identity/v3/users/14b0836d896d4c0eb85eb9f8416a64ac => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:47.839151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bed052-96b3-4bf0-8bba-a02b691c2832 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:47.881727 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab003b41-5b26-4454-9009-edf66060b817 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:47.942611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:47.958054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdabc3-eb35-490a-b300-da1041f29760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:47.959112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdabc3-eb35-490a-b300-da1041f29760 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.959529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdabc3-eb35-490a-b300-da1041f29760 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.959935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdabc3-eb35-490a-b300-da1041f29760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.998685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:48.025655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bed052-96b3-4bf0-8bba-a02b691c2832 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:48.074357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ec00a-1b35-4180-ac6c-4bf15d540932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:48.075340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ec00a-1b35-4180-ac6c-4bf15d540932 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:48.075629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ec00a-1b35-4180-ac6c-4bf15d540932 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:48.075911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ec00a-1b35-4180-ac6c-4bf15d540932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:48.084507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdabc3-eb35-490a-b300-da1041f29760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:48.086349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdabc3-eb35-490a-b300-da1041f29760 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:48.175020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ec00a-1b35-4180-ac6c-4bf15d540932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:48.176503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ec00a-1b35-4180-ac6c-4bf15d540932 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:48.221009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 98/195] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:48.226654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdabc3-eb35-490a-b300-da1041f29760 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:48.252669 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:48.291273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:48.309177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 98/196] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:48.315982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ec00a-1b35-4180-ac6c-4bf15d540932 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:48.348790 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdabc3-eb35-490a-b300-da1041f29760 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:48.389001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63345be4-c6d6-45d5-b3b6-54687bead155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:48.389650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63345be4-c6d6-45d5-b3b6-54687bead155 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:48.389937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63345be4-c6d6-45d5-b3b6-54687bead155 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:48.390298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63345be4-c6d6-45d5-b3b6-54687bead155 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:48.392034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63345be4-c6d6-45d5-b3b6-54687bead155 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:48.393399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63345be4-c6d6-45d5-b3b6-54687bead155 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:48.398873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 99/197] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:48 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:48.401391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63345be4-c6d6-45d5-b3b6-54687bead155 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:48.498635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ec00a-1b35-4180-ac6c-4bf15d540932 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:48.552319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63345be4-c6d6-45d5-b3b6-54687bead155 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:48.557906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b6e6f-fdad-4562-a112-05fd2574c86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:48.558588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b6e6f-fdad-4562-a112-05fd2574c86c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:48.558876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b6e6f-fdad-4562-a112-05fd2574c86c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:48.559157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b6e6f-fdad-4562-a112-05fd2574c86c None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:48.565194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b6e6f-fdad-4562-a112-05fd2574c86c None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:48.566759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b6e6f-fdad-4562-a112-05fd2574c86c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:48.570482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 99/198] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:48 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:48.580681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b6e6f-fdad-4562-a112-05fd2574c86c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:48.600004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:48.622438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:48.630360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4efbe7-ae07-4b84-a53b-e4f80b1a5422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:48.631092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4efbe7-ae07-4b84-a53b-e4f80b1a5422 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:48.631376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4efbe7-ae07-4b84-a53b-e4f80b1a5422 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:48.631652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4efbe7-ae07-4b84-a53b-e4f80b1a5422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:48.632403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4efbe7-ae07-4b84-a53b-e4f80b1a5422 None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4, user.id=14b0836d896d4c0eb85eb9f8416a64ac) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:48.632716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4efbe7-ae07-4b84-a53b-e4f80b1a5422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:48.634185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4efbe7-ae07-4b84-a53b-e4f80b1a5422 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:48.642746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 100/199] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:48 2022] GET /identity/v3/role_assignments?user.id=14b0836d896d4c0eb85eb9f8416a64ac&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=9e904216086e4a06abf9216b0c863f89 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:48.744445 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b6e6f-fdad-4562-a112-05fd2574c86c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:48.796334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c80ac-a4fd-4ebc-8730-b1c808b41083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:48.800615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c80ac-a4fd-4ebc-8730-b1c808b41083 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:48.800615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c80ac-a4fd-4ebc-8730-b1c808b41083 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:48.800615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c80ac-a4fd-4ebc-8730-b1c808b41083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:48.800615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c80ac-a4fd-4ebc-8730-b1c808b41083 None admin] RBAC: Adding query filter params (role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4, user.id=14b0836d896d4c0eb85eb9f8416a64ac) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:48.800615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c80ac-a4fd-4ebc-8730-b1c808b41083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:48.800615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c80ac-a4fd-4ebc-8730-b1c808b41083 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:48.811216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 100/200] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:48 2022] GET /identity/v3/role_assignments?user.id=14b0836d896d4c0eb85eb9f8416a64ac&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:48.880055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27f96c-592c-49d5-9a93-3de603515980 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:48.880358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27f96c-592c-49d5-9a93-3de603515980 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:48.880544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27f96c-592c-49d5-9a93-3de603515980 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:48.882175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 101/201] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:48.888029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199dca0-4dca-4a69-a509-9e9bc5fc4628 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:48.888322 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199dca0-4dca-4a69-a509-9e9bc5fc4628 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:48.888598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199dca0-4dca-4a69-a509-9e9bc5fc4628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:48.914379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-1199dca0-4dca-4a69-a509-9e9bc5fc4628 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:48.915802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1199dca0-4dca-4a69-a509-9e9bc5fc4628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:49.026169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 101/202] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:49.415516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27f96c-592c-49d5-9a93-3de603515980 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:49.591565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27f96c-592c-49d5-9a93-3de603515980 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:49.656421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23fc17-f8e1-4014-a65c-be009ea8564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:49.657481 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23fc17-f8e1-4014-a65c-be009ea8564a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:49.657974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23fc17-f8e1-4014-a65c-be009ea8564a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:49.658438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23fc17-f8e1-4014-a65c-be009ea8564a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:49.677867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23fc17-f8e1-4014-a65c-be009ea8564a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-9c23fc17-f8e1-4014-a65c-be009ea8564a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:49.688096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:49.689013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:49.690287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 102/203] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:49.697047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199dca0-4dca-4a69-a509-9e9bc5fc4628 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:49.736845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:49.837688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199dca0-4dca-4a69-a509-9e9bc5fc4628 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:49.872895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447d4c6-8e4f-4d41-85f8-06d81222c72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:49.873519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447d4c6-8e4f-4d41-85f8-06d81222c72f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:49.873800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447d4c6-8e4f-4d41-85f8-06d81222c72f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:49.874071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447d4c6-8e4f-4d41-85f8-06d81222c72f None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:49.874793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447d4c6-8e4f-4d41-85f8-06d81222c72f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:49.875123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447d4c6-8e4f-4d41-85f8-06d81222c72f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:49.876519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447d4c6-8e4f-4d41-85f8-06d81222c72f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:49.885840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 102/204] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:49.892279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23fc17-f8e1-4014-a65c-be009ea8564a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:49.950595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:50.034415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23fc17-f8e1-4014-a65c-be009ea8564a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:50.094123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b463081-6eeb-4a16-af93-750ff45faa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:50.094805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463081-6eeb-4a16-af93-750ff45faa74 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:50.095100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463081-6eeb-4a16-af93-750ff45faa74 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:50.095414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463081-6eeb-4a16-af93-750ff45faa74 None admin] PATH_INFO: `/v3/users/9565cb05f1204af6af9e9f4780d8cf0b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:50.096594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b463081-6eeb-4a16-af93-750ff45faa74 None admin] RBAC: Authorizing `identity:get_user(user_id=9565cb05f1204af6af9e9f4780d8cf0b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:50.098054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b463081-6eeb-4a16-af93-750ff45faa74 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:50.099861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 103/205] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:49 2022] GET /identity/v3/users/9565cb05f1204af6af9e9f4780d8cf0b => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:50.105937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447d4c6-8e4f-4d41-85f8-06d81222c72f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:50.218279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:50.238820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447d4c6-8e4f-4d41-85f8-06d81222c72f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:50.275809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2a57c-8b05-4cee-bc68-79768718da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:50.276780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2a57c-8b05-4cee-bc68-79768718da45 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:50.277297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2a57c-8b05-4cee-bc68-79768718da45 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:50.277782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2a57c-8b05-4cee-bc68-79768718da45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:50.414059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2a57c-8b05-4cee-bc68-79768718da45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:50.416614 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2a57c-8b05-4cee-bc68-79768718da45 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:50.606976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:50.644172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 103/206] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:50.653518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b463081-6eeb-4a16-af93-750ff45faa74 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:50.834317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b463081-6eeb-4a16-af93-750ff45faa74 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:50.851585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:50.883581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37c4ab-e598-4637-b11f-fec02a6e6486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:50.884489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37c4ab-e598-4637-b11f-fec02a6e6486 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:50.885042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37c4ab-e598-4637-b11f-fec02a6e6486 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:50.885557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37c4ab-e598-4637-b11f-fec02a6e6486 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:50.887314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37c4ab-e598-4637-b11f-fec02a6e6486 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:50.888785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37c4ab-e598-4637-b11f-fec02a6e6486 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:50.891350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 104/207] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:50 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:50.897224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2a57c-8b05-4cee-bc68-79768718da45 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:51.037241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2a57c-8b05-4cee-bc68-79768718da45 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:51.091770 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450e748-a6e4-40f1-93e3-285f2a52f09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:51.092402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450e748-a6e4-40f1-93e3-285f2a52f09b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.092684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450e748-a6e4-40f1-93e3-285f2a52f09b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.092957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450e748-a6e4-40f1-93e3-285f2a52f09b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.093748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450e748-a6e4-40f1-93e3-285f2a52f09b None admin] RBAC: Adding query filter params (role.id=d8a54178fbc44a49ad0df8d85a2adf0b, scope.project.id=86e412e829cf47228db2532cb4c7c4e4, user.id=9565cb05f1204af6af9e9f4780d8cf0b) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:51.094067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450e748-a6e4-40f1-93e3-285f2a52f09b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:51.095605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450e748-a6e4-40f1-93e3-285f2a52f09b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:51.102170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 104/208] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:50 2022] GET /identity/v3/role_assignments?user.id=9565cb05f1204af6af9e9f4780d8cf0b&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=d8a54178fbc44a49ad0df8d85a2adf0b => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:51.164098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedce9a-b37e-4a89-98c5-ef263a205c81 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.164647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedce9a-b37e-4a89-98c5-ef263a205c81 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.165038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedce9a-b37e-4a89-98c5-ef263a205c81 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.167675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 105/209] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:51.176635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc5812-81b5-4b13-ac13-3c49d2f4b6f6 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.177053 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc5812-81b5-4b13-ac13-3c49d2f4b6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.177202 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc5812-81b5-4b13-ac13-3c49d2f4b6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.202068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-91cc5812-81b5-4b13-ac13-3c49d2f4b6f6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:51.203922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-91cc5812-81b5-4b13-ac13-3c49d2f4b6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:51.251736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:51.282903 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 105/210] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:51.353212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf10ab-0f82-48f0-b6ca-88da2b0e79e3 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.353682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf10ab-0f82-48f0-b6ca-88da2b0e79e3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.354173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf10ab-0f82-48f0-b6ca-88da2b0e79e3 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.356926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 106/211] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:51.366798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fe2dc-9575-44b6-9f53-74538ca19ed1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.367284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fe2dc-9575-44b6-9f53-74538ca19ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.367764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fe2dc-9575-44b6-9f53-74538ca19ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.394169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-386fe2dc-9575-44b6-9f53-74538ca19ed1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:51.395407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-386fe2dc-9575-44b6-9f53-74538ca19ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:51.476100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 106/212] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:51.575367 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf10ab-0f82-48f0-b6ca-88da2b0e79e3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:51.713409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf10ab-0f82-48f0-b6ca-88da2b0e79e3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:51.753855 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbed722-caf7-4666-8a44-492817244a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:51.754404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbed722-caf7-4666-8a44-492817244a03 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.754743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbed722-caf7-4666-8a44-492817244a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.755020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbed722-caf7-4666-8a44-492817244a03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.768446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbed722-caf7-4666-8a44-492817244a03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-0cbed722-caf7-4666-8a44-492817244a03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:51.773402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:51.776623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:51.779567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 107/213] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:51.786816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fe2dc-9575-44b6-9f53-74538ca19ed1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:51.898234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fe2dc-9575-44b6-9f53-74538ca19ed1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:51.900638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:51.922895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8173c7-220a-41d4-9a25-94d4c866600b None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.923532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8173c7-220a-41d4-9a25-94d4c866600b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.923890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8173c7-220a-41d4-9a25-94d4c866600b None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.925505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 108/214] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:51.930893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e072c9-0b7c-4ffc-9247-36015d646afc None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.931172 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e072c9-0b7c-4ffc-9247-36015d646afc None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.931632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e072c9-0b7c-4ffc-9247-36015d646afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.934657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4a960-59fc-4264-9186-3084e7489aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:51.935396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a960-59fc-4264-9186-3084e7489aab None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.935782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a960-59fc-4264-9186-3084e7489aab None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.936066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a960-59fc-4264-9186-3084e7489aab None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.936858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4a960-59fc-4264-9186-3084e7489aab None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:51.937176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4a960-59fc-4264-9186-3084e7489aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:51.939637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4a960-59fc-4264-9186-3084e7489aab None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:51.949333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 107/215] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:51.955711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c4e072c9-0b7c-4ffc-9247-36015d646afc None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:51.957604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c4e072c9-0b7c-4ffc-9247-36015d646afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:51.957734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4a960-59fc-4264-9186-3084e7489aab None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:52.009968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:52.057961 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 109/216] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:52.063313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e072c9-0b7c-4ffc-9247-36015d646afc None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:52.121564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4a960-59fc-4264-9186-3084e7489aab None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:52.162307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180a1d-74c3-415f-850c-4a8d6c9b8ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:52.162983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180a1d-74c3-415f-850c-4a8d6c9b8ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.163263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180a1d-74c3-415f-850c-4a8d6c9b8ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.163539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180a1d-74c3-415f-850c-4a8d6c9b8ad0 None admin] PATH_INFO: `/v3/users/f39333e4f0d0425a99da37e75c007b5f` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.193258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78180a1d-74c3-415f-850c-4a8d6c9b8ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=f39333e4f0d0425a99da37e75c007b5f)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:52.194774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78180a1d-74c3-415f-850c-4a8d6c9b8ad0 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:52.196830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 108/217] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:51 2022] GET /identity/v3/users/f39333e4f0d0425a99da37e75c007b5f => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:52.198011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4902d-08e5-4b85-a58d-3dd84a0ea13a None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.198297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4902d-08e5-4b85-a58d-3dd84a0ea13a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.198633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4902d-08e5-4b85-a58d-3dd84a0ea13a None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.201342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 109/218] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:52.202264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4902d-08e5-4b85-a58d-3dd84a0ea13a None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:52.221644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e072c9-0b7c-4ffc-9247-36015d646afc None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:52.236889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:52.268266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301cb5e-ebb6-4b96-b334-1664287cb65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:52.268813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301cb5e-ebb6-4b96-b334-1664287cb65b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.269234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301cb5e-ebb6-4b96-b334-1664287cb65b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.269412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301cb5e-ebb6-4b96-b334-1664287cb65b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.270587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301cb5e-ebb6-4b96-b334-1664287cb65b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:52.272057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301cb5e-ebb6-4b96-b334-1664287cb65b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:52.278683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 110/219] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:52.278683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a838c-be78-4753-86b3-874743f5ebfc None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.278683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a838c-be78-4753-86b3-874743f5ebfc None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.278683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a838c-be78-4753-86b3-874743f5ebfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.299998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7f2a838c-be78-4753-86b3-874743f5ebfc None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:52.301435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7f2a838c-be78-4753-86b3-874743f5ebfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:52.335789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:52.338225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4902d-08e5-4b85-a58d-3dd84a0ea13a None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:52.379009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 111/220] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:52.379844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a838c-be78-4753-86b3-874743f5ebfc None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:52.382917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e31c7-73cb-4693-b0ce-fc44f1a0a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:52.383545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e31c7-73cb-4693-b0ce-fc44f1a0a87c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.383835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e31c7-73cb-4693-b0ce-fc44f1a0a87c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.384113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e31c7-73cb-4693-b0ce-fc44f1a0a87c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.477824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e31c7-73cb-4693-b0ce-fc44f1a0a87c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:52.479345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e31c7-73cb-4693-b0ce-fc44f1a0a87c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:52.508182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:52.511767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a838c-be78-4753-86b3-874743f5ebfc None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:52.553368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29004b1a-fa49-4fb5-ab90-e20062ce5b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:52.554210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29004b1a-fa49-4fb5-ab90-e20062ce5b64 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.554679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29004b1a-fa49-4fb5-ab90-e20062ce5b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.555121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29004b1a-fa49-4fb5-ab90-e20062ce5b64 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.555919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29004b1a-fa49-4fb5-ab90-e20062ce5b64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:52.557547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29004b1a-fa49-4fb5-ab90-e20062ce5b64 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:52.615011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 110/221] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:52.616608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e31c7-73cb-4693-b0ce-fc44f1a0a87c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:52.674540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 112/222] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:29:52 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:52.674540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29004b1a-fa49-4fb5-ab90-e20062ce5b64 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:52.712577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:52.754820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e31c7-73cb-4693-b0ce-fc44f1a0a87c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:52.799362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:52.813787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a4f54-7e35-4839-8d4b-60d94a8bb09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:52.814878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a4f54-7e35-4839-8d4b-60d94a8bb09b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.815305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a4f54-7e35-4839-8d4b-60d94a8bb09b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.820716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a4f54-7e35-4839-8d4b-60d94a8bb09b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.826874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29004b1a-fa49-4fb5-ab90-e20062ce5b64 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:52.831316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a4f54-7e35-4839-8d4b-60d94a8bb09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-5e5a4f54-7e35-4839-8d4b-60d94a8bb09b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:52.837311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:52.842302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 111/223] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:52.847125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a4f54-7e35-4839-8d4b-60d94a8bb09b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:52.886011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb5926-35e4-4c84-841d-3569a405ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:52.887923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb5926-35e4-4c84-841d-3569a405ce8b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.887923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb5926-35e4-4c84-841d-3569a405ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.888221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb5926-35e4-4c84-841d-3569a405ce8b None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.890175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb5926-35e4-4c84-841d-3569a405ce8b None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:52.894754 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb5926-35e4-4c84-841d-3569a405ce8b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:52.896963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 113/224] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:52 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:52.898308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb5926-35e4-4c84-841d-3569a405ce8b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:52.981616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a4f54-7e35-4839-8d4b-60d94a8bb09b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:53.034186 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f7db6-987e-4216-985b-3affe1cd6355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:53.035182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f7db6-987e-4216-985b-3affe1cd6355 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.035648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f7db6-987e-4216-985b-3affe1cd6355 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.036132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f7db6-987e-4216-985b-3affe1cd6355 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.045220 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:53.051335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f7db6-987e-4216-985b-3affe1cd6355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:53.055373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb5926-35e4-4c84-841d-3569a405ce8b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-4c1f7db6-987e-4216-985b-3affe1cd6355 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:53.056935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:53.064216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 112/225] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:29:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:53.065074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f7db6-987e-4216-985b-3affe1cd6355 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:53.106316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3438b-7920-4c5e-9a4f-b888fdc47010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:53.107022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3438b-7920-4c5e-9a4f-b888fdc47010 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.107386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3438b-7920-4c5e-9a4f-b888fdc47010 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.107684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3438b-7920-4c5e-9a4f-b888fdc47010 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.108619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3438b-7920-4c5e-9a4f-b888fdc47010 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:53.108928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3438b-7920-4c5e-9a4f-b888fdc47010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:53.110439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3438b-7920-4c5e-9a4f-b888fdc47010 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:53.122177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:53.123996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 114/226] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:53.125868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3438b-7920-4c5e-9a4f-b888fdc47010 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:53.219141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f7db6-987e-4216-985b-3affe1cd6355 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:53.264657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71573b17-a5a0-4de7-85cb-1c1b27796b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:53.265373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71573b17-a5a0-4de7-85cb-1c1b27796b05 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.265752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71573b17-a5a0-4de7-85cb-1c1b27796b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.266031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71573b17-a5a0-4de7-85cb-1c1b27796b05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.266878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71573b17-a5a0-4de7-85cb-1c1b27796b05 None admin] RBAC: Adding query filter params (role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4, user.id=f39333e4f0d0425a99da37e75c007b5f) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:53.267306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71573b17-a5a0-4de7-85cb-1c1b27796b05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:53.268991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71573b17-a5a0-4de7-85cb-1c1b27796b05 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:53.276752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 113/227] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:53 2022] GET /identity/v3/role_assignments?user.id=f39333e4f0d0425a99da37e75c007b5f&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:53.277600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71573b17-a5a0-4de7-85cb-1c1b27796b05 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:53.292411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3438b-7920-4c5e-9a4f-b888fdc47010 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:53.372953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36befc46-ada9-4875-9545-da7297a3d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:53.373501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36befc46-ada9-4875-9545-da7297a3d655 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.373783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36befc46-ada9-4875-9545-da7297a3d655 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.374162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36befc46-ada9-4875-9545-da7297a3d655 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.376963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36befc46-ada9-4875-9545-da7297a3d655 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:53.377299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36befc46-ada9-4875-9545-da7297a3d655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:53.379044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36befc46-ada9-4875-9545-da7297a3d655 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:53.400974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 115/228] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:53.407244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36befc46-ada9-4875-9545-da7297a3d655 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:53.454899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:53.499700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71573b17-a5a0-4de7-85cb-1c1b27796b05 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:53.519032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:53.550976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709bd22-8c5c-4347-b70d-08d366cc1847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:53.708854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709bd22-8c5c-4347-b70d-08d366cc1847 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.708854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709bd22-8c5c-4347-b70d-08d366cc1847 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.708854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709bd22-8c5c-4347-b70d-08d366cc1847 None admin] PATH_INFO: `/v3/users/14b0836d896d4c0eb85eb9f8416a64ac` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.708854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709bd22-8c5c-4347-b70d-08d366cc1847 None admin] RBAC: Authorizing `identity:get_user(user_id=14b0836d896d4c0eb85eb9f8416a64ac)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:53.708854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709bd22-8c5c-4347-b70d-08d366cc1847 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:53.708854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 114/229] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:53 2022] GET /identity/v3/users/14b0836d896d4c0eb85eb9f8416a64ac => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:53.709280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709bd22-8c5c-4347-b70d-08d366cc1847 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:53.709348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36befc46-ada9-4875-9545-da7297a3d655 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:53.709348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe9ee9-c997-4ed2-bcb2-c35e1dbb6b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:53.709348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe9ee9-c997-4ed2-bcb2-c35e1dbb6b21 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.709348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe9ee9-c997-4ed2-bcb2-c35e1dbb6b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.709604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe9ee9-c997-4ed2-bcb2-c35e1dbb6b21 None admin] PATH_INFO: `/v3/users/14b0836d896d4c0eb85eb9f8416a64ac` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.709604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe9ee9-c997-4ed2-bcb2-c35e1dbb6b21 None admin] RBAC: Authorizing `identity:get_user(user_id=14b0836d896d4c0eb85eb9f8416a64ac)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:53.709604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe9ee9-c997-4ed2-bcb2-c35e1dbb6b21 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:53.709604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 116/230] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:53 2022] GET /identity/v3/users/14b0836d896d4c0eb85eb9f8416a64ac => generated 283 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:53.709604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe9ee9-c997-4ed2-bcb2-c35e1dbb6b21 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:53.766437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709bd22-8c5c-4347-b70d-08d366cc1847 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:53.832966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea046c5f-c940-4261-9f6c-5ff40efefe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:53.833629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea046c5f-c940-4261-9f6c-5ff40efefe21 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.833917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea046c5f-c940-4261-9f6c-5ff40efefe21 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.834200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea046c5f-c940-4261-9f6c-5ff40efefe21 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.835385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea046c5f-c940-4261-9f6c-5ff40efefe21 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:53.837889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea046c5f-c940-4261-9f6c-5ff40efefe21 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:53.839631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 115/231] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:53 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:53.847644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea046c5f-c940-4261-9f6c-5ff40efefe21 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:53.877093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe9ee9-c997-4ed2-bcb2-c35e1dbb6b21 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:53.901764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:53.943540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fdde8-dbdb-4e8f-ba3b-cdec9d49f8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:53.944192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fdde8-dbdb-4e8f-ba3b-cdec9d49f8ec None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fdde8-dbdb-4e8f-ba3b-cdec9d49f8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.944750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fdde8-dbdb-4e8f-ba3b-cdec9d49f8ec None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.946489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fdde8-dbdb-4e8f-ba3b-cdec9d49f8ec None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:53.947972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fdde8-dbdb-4e8f-ba3b-cdec9d49f8ec None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:53.949623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 117/232] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:53 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:53.956317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fdde8-dbdb-4e8f-ba3b-cdec9d49f8ec None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:54.090853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea046c5f-c940-4261-9f6c-5ff40efefe21 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:54.105027 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4de649-651c-49df-9b48-ce68aed77efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:54.105983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de649-651c-49df-9b48-ce68aed77efd None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:54.106470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de649-651c-49df-9b48-ce68aed77efd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:54.107026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de649-651c-49df-9b48-ce68aed77efd None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/users/14b0836d896d4c0eb85eb9f8416a64ac/roles/e5de7bbccf114283851bcc0028d7cc51` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:54.113782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4de649-651c-49df-9b48-ce68aed77efd None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b2151f913144fd856a4dd3b1181fe4, user_id=14b0836d896d4c0eb85eb9f8416a64ac, role_id=e5de7bbccf114283851bcc0028d7cc51)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:54.115556 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4de649-651c-49df-9b48-ce68aed77efd None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:54.119127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:54.133689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fdde8-dbdb-4e8f-ba3b-cdec9d49f8ec None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:54.173770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac373dc3-2965-4766-b4e2-1c05035ffbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:54.175020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac373dc3-2965-4766-b4e2-1c05035ffbc6 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:54.175507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac373dc3-2965-4766-b4e2-1c05035ffbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:54.175982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac373dc3-2965-4766-b4e2-1c05035ffbc6 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/users/14b0836d896d4c0eb85eb9f8416a64ac/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:54.178322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac373dc3-2965-4766-b4e2-1c05035ffbc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b2151f913144fd856a4dd3b1181fe4, user_id=14b0836d896d4c0eb85eb9f8416a64ac, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:54.180115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac373dc3-2965-4766-b4e2-1c05035ffbc6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:54.246354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 116/233] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:29:53 2022] PUT /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/users/14b0836d896d4c0eb85eb9f8416a64ac/roles/e5de7bbccf114283851bcc0028d7cc51 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:29:54.278277 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 118/234] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:29:53 2022] PUT /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/users/14b0836d896d4c0eb85eb9f8416a64ac/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:29:55.114009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe6d96-c192-4e0e-a1d5-d71b41858151 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:55.115299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe6d96-c192-4e0e-a1d5-d71b41858151 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:55.115789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe6d96-c192-4e0e-a1d5-d71b41858151 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:55.117759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 117/235] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:55.138303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec9c1d-0443-44ee-a0e5-c9d51f6f12bf None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:55.139292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec9c1d-0443-44ee-a0e5-c9d51f6f12bf None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:55.139766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec9c1d-0443-44ee-a0e5-c9d51f6f12bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:55.172514 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-56ec9c1d-0443-44ee-a0e5-c9d51f6f12bf None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:55.174774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-56ec9c1d-0443-44ee-a0e5-c9d51f6f12bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:55.189442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:55.287178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 119/236] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:55.463060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926092-252e-4c02-adea-70c351fb98c3 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:55.463735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926092-252e-4c02-adea-70c351fb98c3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:55.464222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926092-252e-4c02-adea-70c351fb98c3 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:55.466017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 118/237] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:55.481300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4a672-1d5a-4625-acac-c52e4e3d132a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:55.481813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4a672-1d5a-4625-acac-c52e4e3d132a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:55.482320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4a672-1d5a-4625-acac-c52e4e3d132a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:55.508018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7fe4a672-1d5a-4625-acac-c52e4e3d132a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:55.509767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7fe4a672-1d5a-4625-acac-c52e4e3d132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:55.608816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 120/238] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:55.731671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157fc6f-a672-4804-ba66-62184a957080 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:55.732100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157fc6f-a672-4804-ba66-62184a957080 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:55.732546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157fc6f-a672-4804-ba66-62184a957080 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:55.735471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 119/239] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:55.742955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad53b6-7fca-4e3a-a493-420565db5378 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:55.743245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad53b6-7fca-4e3a-a493-420565db5378 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:55.743523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad53b6-7fca-4e3a-a493-420565db5378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:55.761959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-90ad53b6-7fca-4e3a-a493-420565db5378 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:55.763195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-90ad53b6-7fca-4e3a-a493-420565db5378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:55.794746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157fc6f-a672-4804-ba66-62184a957080 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:55.880936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 121/240] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:29:55.974257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157fc6f-a672-4804-ba66-62184a957080 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:55.976915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:56.048043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5b857-5162-4d04-9a9a-4d4e70ee029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:56.048906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5b857-5162-4d04-9a9a-4d4e70ee029d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:56.049317 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5b857-5162-4d04-9a9a-4d4e70ee029d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:56.049733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5b857-5162-4d04-9a9a-4d4e70ee029d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:56.071933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5b857-5162-4d04-9a9a-4d4e70ee029d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-1da5b857-5162-4d04-9a9a-4d4e70ee029d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:56.078650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:56.079565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:29:56.080515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 120/241] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:29:56.086149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad53b6-7fca-4e3a-a493-420565db5378 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:56.089258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:56.125297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5b857-5162-4d04-9a9a-4d4e70ee029d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:56.227388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad53b6-7fca-4e3a-a493-420565db5378 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:56.276326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f11eab-051e-40ea-a78b-456c454fcdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:56.277296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f11eab-051e-40ea-a78b-456c454fcdee None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:56.277592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f11eab-051e-40ea-a78b-456c454fcdee None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:56.277869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f11eab-051e-40ea-a78b-456c454fcdee None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:56.278749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f11eab-051e-40ea-a78b-456c454fcdee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:56.279187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f11eab-051e-40ea-a78b-456c454fcdee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:56.281707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f11eab-051e-40ea-a78b-456c454fcdee None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:56.295530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 122/242] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:56.302533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f11eab-051e-40ea-a78b-456c454fcdee None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:56.310745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5b857-5162-4d04-9a9a-4d4e70ee029d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:56.358094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b8f52-ab6e-4ea2-9a69-aa6e58dd83c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:56.359359 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b8f52-ab6e-4ea2-9a69-aa6e58dd83c7 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:56.359652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b8f52-ab6e-4ea2-9a69-aa6e58dd83c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:56.359928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b8f52-ab6e-4ea2-9a69-aa6e58dd83c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:56.361313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b8f52-ab6e-4ea2-9a69-aa6e58dd83c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:56.363122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b8f52-ab6e-4ea2-9a69-aa6e58dd83c7 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:56.365996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 121/243] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:29:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:56.366829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b8f52-ab6e-4ea2-9a69-aa6e58dd83c7 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:56.404396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:56.411601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:56.462574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f11eab-051e-40ea-a78b-456c454fcdee None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:56.517707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80047f88-e2e4-498f-9cac-46aa266a4646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:56.518647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80047f88-e2e4-498f-9cac-46aa266a4646 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:56.518935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80047f88-e2e4-498f-9cac-46aa266a4646 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:56.519211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80047f88-e2e4-498f-9cac-46aa266a4646 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:56.522296 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b8f52-ab6e-4ea2-9a69-aa6e58dd83c7 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:56.530721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80047f88-e2e4-498f-9cac-46aa266a4646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-80047f88-e2e4-498f-9cac-46aa266a4646 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:56.537073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:56.538004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:56.538925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 123/244] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:56.539593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80047f88-e2e4-498f-9cac-46aa266a4646 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:56.568882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a630e-d423-42a2-b667-7da74f2d1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:56.571693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a630e-d423-42a2-b667-7da74f2d1784 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:56.572207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a630e-d423-42a2-b667-7da74f2d1784 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:56.572694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a630e-d423-42a2-b667-7da74f2d1784 None admin] PATH_INFO: `/v3/users/ab407f2ac3c44e6285a7add6d54e806d` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:56.589856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a630e-d423-42a2-b667-7da74f2d1784 None admin] RBAC: Authorizing `identity:get_user(user_id=ab407f2ac3c44e6285a7add6d54e806d)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:56.591718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a630e-d423-42a2-b667-7da74f2d1784 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:56.594331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 122/245] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:56 2022] GET /identity/v3/users/ab407f2ac3c44e6285a7add6d54e806d => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:56.595811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a630e-d423-42a2-b667-7da74f2d1784 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:56.676339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80047f88-e2e4-498f-9cac-46aa266a4646 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:56.701809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:56.708208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:56.714215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a630e-d423-42a2-b667-7da74f2d1784 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:56.721327 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a809-e420-45d6-a640-6ca17c2721c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:56.722036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a809-e420-45d6-a640-6ca17c2721c3 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:56.722437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a809-e420-45d6-a640-6ca17c2721c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:56.722837 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a809-e420-45d6-a640-6ca17c2721c3 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:56.723793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351a809-e420-45d6-a640-6ca17c2721c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:56.725474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351a809-e420-45d6-a640-6ca17c2721c3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:56.755548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac33c1-c4b4-4887-920f-e8868110c73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:56.756174 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac33c1-c4b4-4887-920f-e8868110c73d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:56.756462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac33c1-c4b4-4887-920f-e8868110c73d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:56.756746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac33c1-c4b4-4887-920f-e8868110c73d None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:56.757425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac33c1-c4b4-4887-920f-e8868110c73d None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:56.757740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac33c1-c4b4-4887-920f-e8868110c73d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:56.759185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac33c1-c4b4-4887-920f-e8868110c73d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:56.769649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 123/246] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:29:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:56.770255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac33c1-c4b4-4887-920f-e8868110c73d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:56.850452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 124/247] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:29:56 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:29:56.852500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a809-e420-45d6-a640-6ca17c2721c3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:56.949284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac33c1-c4b4-4887-920f-e8868110c73d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:57.017613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766a232-6133-4fc3-8438-66a056983d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:57.019086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766a232-6133-4fc3-8438-66a056983d8e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.020779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766a232-6133-4fc3-8438-66a056983d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.022565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766a232-6133-4fc3-8438-66a056983d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.039220 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a809-e420-45d6-a640-6ca17c2721c3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:57.070979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:57.095183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b66c15-79a3-4f37-bf98-e9a740fe5b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:57.097201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b66c15-79a3-4f37-bf98-e9a740fe5b56 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.097701 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b66c15-79a3-4f37-bf98-e9a740fe5b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.098265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b66c15-79a3-4f37-bf98-e9a740fe5b56 None admin] PATH_INFO: `/v3/users/f39333e4f0d0425a99da37e75c007b5f` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.099854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b66c15-79a3-4f37-bf98-e9a740fe5b56 None admin] RBAC: Authorizing `identity:get_user(user_id=f39333e4f0d0425a99da37e75c007b5f)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:57.101380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b66c15-79a3-4f37-bf98-e9a740fe5b56 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:57.103130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 125/248] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:56 2022] GET /identity/v3/users/f39333e4f0d0425a99da37e75c007b5f => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:57.104323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35885-37a5-42fb-8b51-b8010d185cc6 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.104599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35885-37a5-42fb-8b51-b8010d185cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.104937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35885-37a5-42fb-8b51-b8010d185cc6 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.106349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 126/249] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:57.107827 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd52f1-4f5d-4cec-9461-e6ad5e42b8ec None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.108115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd52f1-4f5d-4cec-9461-e6ad5e42b8ec None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.108389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd52f1-4f5d-4cec-9461-e6ad5e42b8ec None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.111010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 127/250] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:57.111801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd52f1-4f5d-4cec-9461-e6ad5e42b8ec None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:57.149483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766a232-6133-4fc3-8438-66a056983d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:57.151038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766a232-6133-4fc3-8438-66a056983d8e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:57.204437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:57.311711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd52f1-4f5d-4cec-9461-e6ad5e42b8ec None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:57.315956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 124/251] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:29:57.318349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b9604-4ab0-4041-ac50-1710f3fbf4b8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.319024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b9604-4ab0-4041-ac50-1710f3fbf4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.319655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b9604-4ab0-4041-ac50-1710f3fbf4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.351135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-0c5b9604-4ab0-4041-ac50-1710f3fbf4b8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:57.353385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0c5b9604-4ab0-4041-ac50-1710f3fbf4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:57.368116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940b92c-e6f1-4162-a291-458c0f4f8df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:57.369106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940b92c-e6f1-4162-a291-458c0f4f8df1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.369576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940b92c-e6f1-4162-a291-458c0f4f8df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.370045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940b92c-e6f1-4162-a291-458c0f4f8df1 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.371904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940b92c-e6f1-4162-a291-458c0f4f8df1 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:57.374437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940b92c-e6f1-4162-a291-458c0f4f8df1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:57.382664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 128/252] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:57 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:57.384769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e228996-016c-4c87-887c-ebd83200b787 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.385241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e228996-016c-4c87-887c-ebd83200b787 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.385713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e228996-016c-4c87-887c-ebd83200b787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.417006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-0e228996-016c-4c87-887c-ebd83200b787 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:57.418480 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0e228996-016c-4c87-887c-ebd83200b787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:57.441337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:57.481229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 125/253] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:57.485135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b9604-4ab0-4041-ac50-1710f3fbf4b8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:57.537899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 129/254] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:29:57.538705 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e228996-016c-4c87-887c-ebd83200b787 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:57.632490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:57.657802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b9604-4ab0-4041-ac50-1710f3fbf4b8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:57.725089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd537cd-4d49-4464-ab3c-269ff1d4ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:57.728110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd537cd-4d49-4464-ab3c-269ff1d4ddde None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.728824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd537cd-4d49-4464-ab3c-269ff1d4ddde None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.730311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd537cd-4d49-4464-ab3c-269ff1d4ddde None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.731966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd537cd-4d49-4464-ab3c-269ff1d4ddde None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:57.737919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e228996-016c-4c87-887c-ebd83200b787 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:57.738620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd537cd-4d49-4464-ab3c-269ff1d4ddde None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:57.742363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 126/255] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:57 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:57.749534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd537cd-4d49-4464-ab3c-269ff1d4ddde None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:57.823564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77a866-d4cc-4ece-8f9b-748f596398b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:57.824645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77a866-d4cc-4ece-8f9b-748f596398b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.825122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77a866-d4cc-4ece-8f9b-748f596398b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.825589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77a866-d4cc-4ece-8f9b-748f596398b7 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/users/f39333e4f0d0425a99da37e75c007b5f/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.828931 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77a866-d4cc-4ece-8f9b-748f596398b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b2151f913144fd856a4dd3b1181fe4, user_id=f39333e4f0d0425a99da37e75c007b5f, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:57.830614 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77a866-d4cc-4ece-8f9b-748f596398b7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:57.846780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:57.910696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 130/256] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:29:57 2022] PUT /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/users/f39333e4f0d0425a99da37e75c007b5f/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:29:57.927288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd537cd-4d49-4464-ab3c-269ff1d4ddde None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:57.980789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89f98d-5913-45ec-ac46-2835261fac33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:57.981705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89f98d-5913-45ec-ac46-2835261fac33 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.982191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89f98d-5913-45ec-ac46-2835261fac33 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.982674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89f98d-5913-45ec-ac46-2835261fac33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.983867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89f98d-5913-45ec-ac46-2835261fac33 None admin] RBAC: Adding query filter params (role.id=d8a54178fbc44a49ad0df8d85a2adf0b, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4, user.id=ab407f2ac3c44e6285a7add6d54e806d) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:57.985006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89f98d-5913-45ec-ac46-2835261fac33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:57.986778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89f98d-5913-45ec-ac46-2835261fac33 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:57.993768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 127/257] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:57 2022] GET /identity/v3/role_assignments?user.id=ab407f2ac3c44e6285a7add6d54e806d&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=d8a54178fbc44a49ad0df8d85a2adf0b => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:58.032231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77a866-d4cc-4ece-8f9b-748f596398b7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:58.198298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77a866-d4cc-4ece-8f9b-748f596398b7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:58.210178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:58.245645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced33848-5793-44b6-8e9d-5aa86c6746ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:58.246727 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33848-5793-44b6-8e9d-5aa86c6746ca None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:58.247012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33848-5793-44b6-8e9d-5aa86c6746ca None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:58.247344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33848-5793-44b6-8e9d-5aa86c6746ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:58.267087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced33848-5793-44b6-8e9d-5aa86c6746ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-ced33848-5793-44b6-8e9d-5aa86c6746ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:58.277163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:58.278059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:29:58.278914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:58.280038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 131/258] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:29:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:29:58.287748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89f98d-5913-45ec-ac46-2835261fac33 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:58.396580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced33848-5793-44b6-8e9d-5aa86c6746ca None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:58.422952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89f98d-5913-45ec-ac46-2835261fac33 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:58.457597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:58.527578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98fe9f-a6a9-4679-b6f4-2422c56bb483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:58.528676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98fe9f-a6a9-4679-b6f4-2422c56bb483 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:58.529237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98fe9f-a6a9-4679-b6f4-2422c56bb483 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:58.529783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98fe9f-a6a9-4679-b6f4-2422c56bb483 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:58.531018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98fe9f-a6a9-4679-b6f4-2422c56bb483 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:58.531598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98fe9f-a6a9-4679-b6f4-2422c56bb483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:58.534180 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98fe9f-a6a9-4679-b6f4-2422c56bb483 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:58.566049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 128/259] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:29:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:58.571959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98fe9f-a6a9-4679-b6f4-2422c56bb483 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:58.682733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced33848-5793-44b6-8e9d-5aa86c6746ca None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:58.730760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c411593-240e-4745-94c9-b52e9190693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:58.731281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c411593-240e-4745-94c9-b52e9190693e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:58.731567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c411593-240e-4745-94c9-b52e9190693e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:58.731845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c411593-240e-4745-94c9-b52e9190693e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:58.747703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c411593-240e-4745-94c9-b52e9190693e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-7c411593-240e-4745-94c9-b52e9190693e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:29:58.754284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:29:58.755234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:29:58.756049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:29:58.762360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 132/260] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:29:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:29:58.772773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c411593-240e-4745-94c9-b52e9190693e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:58.819116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:58.834847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98fe9f-a6a9-4679-b6f4-2422c56bb483 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:58.887165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c16e4-6a56-4aa1-aeea-053f6de3db0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:58.890628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c16e4-6a56-4aa1-aeea-053f6de3db0a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:58.890628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c16e4-6a56-4aa1-aeea-053f6de3db0a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:58.890628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c16e4-6a56-4aa1-aeea-053f6de3db0a None admin] PATH_INFO: `/v3/users/14b0836d896d4c0eb85eb9f8416a64ac` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:58.892171 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c16e4-6a56-4aa1-aeea-053f6de3db0a None admin] RBAC: Authorizing `identity:get_user(user_id=14b0836d896d4c0eb85eb9f8416a64ac)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:58.898655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c16e4-6a56-4aa1-aeea-053f6de3db0a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:58.901083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 129/261] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:58 2022] GET /identity/v3/users/14b0836d896d4c0eb85eb9f8416a64ac => generated 283 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:58.906704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c16e4-6a56-4aa1-aeea-053f6de3db0a None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:58.952408 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c411593-240e-4745-94c9-b52e9190693e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:58.990370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:59.018588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c781f38-5ab7-4bac-bee7-6c45b38d34dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:59.019392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c781f38-5ab7-4bac-bee7-6c45b38d34dc None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:59.019687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c781f38-5ab7-4bac-bee7-6c45b38d34dc None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:59.020009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c781f38-5ab7-4bac-bee7-6c45b38d34dc None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:59.021531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c781f38-5ab7-4bac-bee7-6c45b38d34dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:29:59.021845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c781f38-5ab7-4bac-bee7-6c45b38d34dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:59.024024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c781f38-5ab7-4bac-bee7-6c45b38d34dc None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:59.049312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 133/262] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:29:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:29:59.057656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c781f38-5ab7-4bac-bee7-6c45b38d34dc None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:59.111771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c16e4-6a56-4aa1-aeea-053f6de3db0a None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:59.162162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e913b9-2efb-437b-b154-204f4ebb11fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:59.163100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e913b9-2efb-437b-b154-204f4ebb11fa None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:59.163485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e913b9-2efb-437b-b154-204f4ebb11fa None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:59.163907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e913b9-2efb-437b-b154-204f4ebb11fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:59.232400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:59.260509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c781f38-5ab7-4bac-bee7-6c45b38d34dc None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:59.271477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e913b9-2efb-437b-b154-204f4ebb11fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:59.274812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e913b9-2efb-437b-b154-204f4ebb11fa None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:59.332582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:59.351825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328c75c-f0ba-48ed-a560-33834bbf5dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:59.352484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328c75c-f0ba-48ed-a560-33834bbf5dbc None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:59.352766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328c75c-f0ba-48ed-a560-33834bbf5dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:59.353044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328c75c-f0ba-48ed-a560-33834bbf5dbc None admin] PATH_INFO: `/v3/users/14b0836d896d4c0eb85eb9f8416a64ac` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:59.355599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328c75c-f0ba-48ed-a560-33834bbf5dbc None admin] RBAC: Authorizing `identity:get_user(user_id=14b0836d896d4c0eb85eb9f8416a64ac)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:59.357073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328c75c-f0ba-48ed-a560-33834bbf5dbc None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:59.359383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 134/263] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:29:59 2022] GET /identity/v3/users/14b0836d896d4c0eb85eb9f8416a64ac => generated 283 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:59.364900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328c75c-f0ba-48ed-a560-33834bbf5dbc None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:59.401501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 130/264] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:29:59.416241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e913b9-2efb-437b-b154-204f4ebb11fa None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:59.617552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328c75c-f0ba-48ed-a560-33834bbf5dbc None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:59.681223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e913b9-2efb-437b-b154-204f4ebb11fa None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:59.698292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd440ed-67cb-4cbd-9d8b-4f35dc262234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:59.698292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd440ed-67cb-4cbd-9d8b-4f35dc262234 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:59.698292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd440ed-67cb-4cbd-9d8b-4f35dc262234 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:59.698292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd440ed-67cb-4cbd-9d8b-4f35dc262234 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:59.734671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:59.759356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43152247-7d6f-476e-97dd-466e242744fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:59.760357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43152247-7d6f-476e-97dd-466e242744fc None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:59.760832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43152247-7d6f-476e-97dd-466e242744fc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:59.761316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43152247-7d6f-476e-97dd-466e242744fc None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:59.765403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43152247-7d6f-476e-97dd-466e242744fc None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:59.769795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43152247-7d6f-476e-97dd-466e242744fc None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:59.772203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 131/265] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:29:59 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:29:59.778061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43152247-7d6f-476e-97dd-466e242744fc None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:59.803091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:59.811069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd440ed-67cb-4cbd-9d8b-4f35dc262234 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:59.813171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd440ed-67cb-4cbd-9d8b-4f35dc262234 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:59.978834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43152247-7d6f-476e-97dd-466e242744fc None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:00.038952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 135/266] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:30:00.040242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50380817-a141-424b-ba0e-c0416ed364b5 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:00.040534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50380817-a141-424b-ba0e-c0416ed364b5 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:00.040815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50380817-a141-424b-ba0e-c0416ed364b5 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:00.042095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 136/267] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:00.048337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50380817-a141-424b-ba0e-c0416ed364b5 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:00.065946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf41a78-6e89-41ed-95b7-9d570f3806eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:00.066601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41a78-6e89-41ed-95b7-9d570f3806eb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:00.066893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41a78-6e89-41ed-95b7-9d570f3806eb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:00.069784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41a78-6e89-41ed-95b7-9d570f3806eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:00.070580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41a78-6e89-41ed-95b7-9d570f3806eb None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4, user.id=14b0836d896d4c0eb85eb9f8416a64ac) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:00.070907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41a78-6e89-41ed-95b7-9d570f3806eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:00.074105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41a78-6e89-41ed-95b7-9d570f3806eb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:00.086405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 132/268] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:29:59 2022] GET /identity/v3/role_assignments?user.id=14b0836d896d4c0eb85eb9f8416a64ac&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=9e904216086e4a06abf9216b0c863f89 => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:00.093066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdb9ec-4447-428a-b786-6b8716432d7d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:00.093410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdb9ec-4447-428a-b786-6b8716432d7d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:00.093776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdb9ec-4447-428a-b786-6b8716432d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:00.138938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-abcdb9ec-4447-428a-b786-6b8716432d7d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:00.140237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-abcdb9ec-4447-428a-b786-6b8716432d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:00.204744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:00.236818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 133/269] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:00.300092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50380817-a141-424b-ba0e-c0416ed364b5 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:00.345219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f8b1b-c5d6-440c-ac27-a92be1289b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:00.345881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f8b1b-c5d6-440c-ac27-a92be1289b83 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:00.346165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f8b1b-c5d6-440c-ac27-a92be1289b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:00.346439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f8b1b-c5d6-440c-ac27-a92be1289b83 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:00.347695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f8b1b-c5d6-440c-ac27-a92be1289b83 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:00.349185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f8b1b-c5d6-440c-ac27-a92be1289b83 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:00.351060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 137/270] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:00 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:00.360270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdb9ec-4447-428a-b786-6b8716432d7d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:00.382883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:00.425945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740fa8e-7df0-4a05-abb3-4cd21c946f95 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:00.426411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740fa8e-7df0-4a05-abb3-4cd21c946f95 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:00.426843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740fa8e-7df0-4a05-abb3-4cd21c946f95 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:00.429536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 138/271] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:00.433813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204611f-5777-4835-8dca-a088600c1646 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:00.434107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204611f-5777-4835-8dca-a088600c1646 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:00.434376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204611f-5777-4835-8dca-a088600c1646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:00.457598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2204611f-5777-4835-8dca-a088600c1646 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:00.458856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2204611f-5777-4835-8dca-a088600c1646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:00.501434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdb9ec-4447-428a-b786-6b8716432d7d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:00.541871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd926415-1c8a-4e29-a8de-e4a70bc0a196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:00.542852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd926415-1c8a-4e29-a8de-e4a70bc0a196 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:00.542852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd926415-1c8a-4e29-a8de-e4a70bc0a196 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:00.543123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd926415-1c8a-4e29-a8de-e4a70bc0a196 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:00.543849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd926415-1c8a-4e29-a8de-e4a70bc0a196 None admin] RBAC: Adding query filter params (role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4, user.id=14b0836d896d4c0eb85eb9f8416a64ac) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:00.544167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd926415-1c8a-4e29-a8de-e4a70bc0a196 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:00.545705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd926415-1c8a-4e29-a8de-e4a70bc0a196 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:00.546625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 139/272] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:30:00.553840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 134/273] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:00 2022] GET /identity/v3/role_assignments?user.id=14b0836d896d4c0eb85eb9f8416a64ac&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:00.737321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204611f-5777-4835-8dca-a088600c1646 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:00.829437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:00.879347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204611f-5777-4835-8dca-a088600c1646 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:00.917928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfdff6-cf27-41c5-9715-51dd3e142d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:00.918640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfdff6-cf27-41c5-9715-51dd3e142d1b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:00.918887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfdff6-cf27-41c5-9715-51dd3e142d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:00.919161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfdff6-cf27-41c5-9715-51dd3e142d1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:00.920874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfdff6-cf27-41c5-9715-51dd3e142d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:00.924033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfdff6-cf27-41c5-9715-51dd3e142d1b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:00.926233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 140/274] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:00.932089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd926415-1c8a-4e29-a8de-e4a70bc0a196 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:01.027038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:01.046230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfdff6-cf27-41c5-9715-51dd3e142d1b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:01.066890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd926415-1c8a-4e29-a8de-e4a70bc0a196 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:01.116012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e40f52-2f97-4cc7-9d46-4031af758756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:01.116559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e40f52-2f97-4cc7-9d46-4031af758756 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.116846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e40f52-2f97-4cc7-9d46-4031af758756 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.117126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e40f52-2f97-4cc7-9d46-4031af758756 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.117787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e40f52-2f97-4cc7-9d46-4031af758756 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:01.120208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e40f52-2f97-4cc7-9d46-4031af758756 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:01.176470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfdff6-cf27-41c5-9715-51dd3e142d1b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:01.202548 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:01.214546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63d683-fc66-451b-a920-46a048ee1683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:01.215298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63d683-fc66-451b-a920-46a048ee1683 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.215535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63d683-fc66-451b-a920-46a048ee1683 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.215811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63d683-fc66-451b-a920-46a048ee1683 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.230278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 135/275] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:30:00 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:01.235019 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63d683-fc66-451b-a920-46a048ee1683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-ef63d683-fc66-451b-a920-46a048ee1683 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:01.237446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:01.238388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:01.239287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 141/276] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:01.244973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e40f52-2f97-4cc7-9d46-4031af758756 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:01.247535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1326a9-b350-4265-ad9b-ce7dd2beb625 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.247831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1326a9-b350-4265-ad9b-ce7dd2beb625 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.248112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1326a9-b350-4265-ad9b-ce7dd2beb625 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.249552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 142/277] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:01.255684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c77bf-ac60-4286-894a-845257fb51cc None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.255977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c77bf-ac60-4286-894a-845257fb51cc None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.256250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c77bf-ac60-4286-894a-845257fb51cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.276653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-bf0c77bf-ac60-4286-894a-845257fb51cc None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:01.278016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bf0c77bf-ac60-4286-894a-845257fb51cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:01.375737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 143/278] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:01.419383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e40f52-2f97-4cc7-9d46-4031af758756 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:01.449405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:01.471432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025127e-f1c3-4b78-9638-f1c8a07fd432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:01.472108 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025127e-f1c3-4b78-9638-f1c8a07fd432 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.472393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025127e-f1c3-4b78-9638-f1c8a07fd432 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.472668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025127e-f1c3-4b78-9638-f1c8a07fd432 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.473422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025127e-f1c3-4b78-9638-f1c8a07fd432 None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:01.473727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025127e-f1c3-4b78-9638-f1c8a07fd432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:01.475281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025127e-f1c3-4b78-9638-f1c8a07fd432 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:01.486115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 136/279] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:01.491371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c77bf-ac60-4286-894a-845257fb51cc None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:01.599782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bf0dd-672c-4df4-af1c-ba9608bc7d8e None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.600069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bf0dd-672c-4df4-af1c-ba9608bc7d8e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.603042 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bf0dd-672c-4df4-af1c-ba9608bc7d8e None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.604576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 137/280] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:01.611651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506febb-592e-4c05-968d-1c5d7e544777 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.611945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506febb-592e-4c05-968d-1c5d7e544777 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.612213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506febb-592e-4c05-968d-1c5d7e544777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.634865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c77bf-ac60-4286-894a-845257fb51cc None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:01.641756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5506febb-592e-4c05-968d-1c5d7e544777 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:01.643137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5506febb-592e-4c05-968d-1c5d7e544777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:01.653673 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:01.686888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f492-84d8-47a6-a359-35fb4354d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:01.688533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f492-84d8-47a6-a359-35fb4354d019 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.688819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f492-84d8-47a6-a359-35fb4354d019 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.689218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f492-84d8-47a6-a359-35fb4354d019 None admin] PATH_INFO: `/v3/users/3d89c259abf34407bf5daab2a3f1fb47` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.711690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3f492-84d8-47a6-a359-35fb4354d019 None admin] RBAC: Authorizing `identity:get_user(user_id=3d89c259abf34407bf5daab2a3f1fb47)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:01.713333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3f492-84d8-47a6-a359-35fb4354d019 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:01.715261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 144/281] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:30:01 2022] GET /identity/v3/users/3d89c259abf34407bf5daab2a3f1fb47 => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:01.716135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f492-84d8-47a6-a359-35fb4354d019 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:01.744229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 138/282] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:01.745130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506febb-592e-4c05-968d-1c5d7e544777 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:01.881781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506febb-592e-4c05-968d-1c5d7e544777 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:01.904303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:01.920718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f492-84d8-47a6-a359-35fb4354d019 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:01.925334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24bd9a-25ab-4815-82f8-a7046035c8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:01.926233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24bd9a-25ab-4815-82f8-a7046035c8ad None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.926760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24bd9a-25ab-4815-82f8-a7046035c8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.927235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24bd9a-25ab-4815-82f8-a7046035c8ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.929489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24bd9a-25ab-4815-82f8-a7046035c8ad None admin] RBAC: Adding query filter params (role.id=5bf6686ae6624e84aba335589d8a98a4, user.id=3d89c259abf34407bf5daab2a3f1fb47, scope.system=all) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:01.930191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24bd9a-25ab-4815-82f8-a7046035c8ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:01.932688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24bd9a-25ab-4815-82f8-a7046035c8ad None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:01.942132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 139/283] 10.222.0.82 () {56 vars in 1308 bytes} [Wed Aug 24 03:30:01 2022] GET /identity/v3/role_assignments?user.id=3d89c259abf34407bf5daab2a3f1fb47&scope.system=all&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:01.978522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9690f4-3882-461f-a4ee-2aa04dd6fce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:01.979225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9690f4-3882-461f-a4ee-2aa04dd6fce1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:01.979511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9690f4-3882-461f-a4ee-2aa04dd6fce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:01.979789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9690f4-3882-461f-a4ee-2aa04dd6fce1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:01.993891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9690f4-3882-461f-a4ee-2aa04dd6fce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-1f9690f4-3882-461f-a4ee-2aa04dd6fce1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:01.999122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:02.000749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:02.002632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 145/284] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:02.009172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24bd9a-25ab-4815-82f8-a7046035c8ad None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:02.114645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9690f4-3882-461f-a4ee-2aa04dd6fce1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:02.183231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24bd9a-25ab-4815-82f8-a7046035c8ad None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:02.194266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:02.247405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d817b4d-1392-4f70-b90f-24c0cee8f997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:02.249180 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d817b4d-1392-4f70-b90f-24c0cee8f997 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:02.249868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d817b4d-1392-4f70-b90f-24c0cee8f997 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:02.251529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d817b4d-1392-4f70-b90f-24c0cee8f997 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:02.252941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d817b4d-1392-4f70-b90f-24c0cee8f997 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:02.253711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d817b4d-1392-4f70-b90f-24c0cee8f997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:02.256350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d817b4d-1392-4f70-b90f-24c0cee8f997 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:02.268619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 140/285] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:02.275074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d817b4d-1392-4f70-b90f-24c0cee8f997 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:02.336706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:02.357080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9690f4-3882-461f-a4ee-2aa04dd6fce1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:02.405647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d817b4d-1392-4f70-b90f-24c0cee8f997 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:02.444134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bb2df-a459-402d-919d-b15e8fdce0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:02.444708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bb2df-a459-402d-919d-b15e8fdce0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:02.444993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bb2df-a459-402d-919d-b15e8fdce0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:02.445303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bb2df-a459-402d-919d-b15e8fdce0c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:02.449092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279e7f7-61f9-4be3-9938-327636e58ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:02.450320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279e7f7-61f9-4be3-9938-327636e58ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:02.450906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279e7f7-61f9-4be3-9938-327636e58ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:02.451383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279e7f7-61f9-4be3-9938-327636e58ad4 None admin] PATH_INFO: `/v3/users/ab407f2ac3c44e6285a7add6d54e806d` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:02.452898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279e7f7-61f9-4be3-9938-327636e58ad4 None admin] RBAC: Authorizing `identity:get_user(user_id=ab407f2ac3c44e6285a7add6d54e806d)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:02.456435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279e7f7-61f9-4be3-9938-327636e58ad4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:02.458517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 141/286] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:30:02 2022] GET /identity/v3/users/ab407f2ac3c44e6285a7add6d54e806d => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:02.464147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279e7f7-61f9-4be3-9938-327636e58ad4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:02.472377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97bb2df-a459-402d-919d-b15e8fdce0c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-e97bb2df-a459-402d-919d-b15e8fdce0c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:02.477331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:02.478278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:02.479179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 146/287] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:02.485278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bb2df-a459-402d-919d-b15e8fdce0c3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:02.643535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279e7f7-61f9-4be3-9938-327636e58ad4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:02.670051 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:02.671619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:02.689628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bb2df-a459-402d-919d-b15e8fdce0c3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:02.711152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122710b-f355-4637-b7ae-a792585b3387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:02.717718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122710b-f355-4637-b7ae-a792585b3387 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:02.717718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122710b-f355-4637-b7ae-a792585b3387 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:02.717718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122710b-f355-4637-b7ae-a792585b3387 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:02.722530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122710b-f355-4637-b7ae-a792585b3387 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:02.722530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122710b-f355-4637-b7ae-a792585b3387 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:02.725035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 142/288] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:02 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:02.730590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122710b-f355-4637-b7ae-a792585b3387 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:02.758067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9a094-8206-4ee2-a945-887eedc8197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:02.758719 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9a094-8206-4ee2-a945-887eedc8197b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:02.758998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9a094-8206-4ee2-a945-887eedc8197b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:02.759271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9a094-8206-4ee2-a945-887eedc8197b None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:02.760053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9a094-8206-4ee2-a945-887eedc8197b None admin] RBAC: Adding query filter params (name=member) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:02.760394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9a094-8206-4ee2-a945-887eedc8197b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:02.761841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9a094-8206-4ee2-a945-887eedc8197b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:02.780886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 147/289] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:02.805061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9a094-8206-4ee2-a945-887eedc8197b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:02.942267 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122710b-f355-4637-b7ae-a792585b3387 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.008364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822472b6-1f6d-48dc-8276-80e2edf484d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.009160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822472b6-1f6d-48dc-8276-80e2edf484d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.009446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822472b6-1f6d-48dc-8276-80e2edf484d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.009919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822472b6-1f6d-48dc-8276-80e2edf484d4 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/users/ab407f2ac3c44e6285a7add6d54e806d/roles/d8a54178fbc44a49ad0df8d85a2adf0b` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.012290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822472b6-1f6d-48dc-8276-80e2edf484d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b2151f913144fd856a4dd3b1181fe4, user_id=ab407f2ac3c44e6285a7add6d54e806d, role_id=d8a54178fbc44a49ad0df8d85a2adf0b)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.022627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822472b6-1f6d-48dc-8276-80e2edf484d4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.027511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9a094-8206-4ee2-a945-887eedc8197b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.066104 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacb350-ef61-43bc-925d-6ac31b37563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.066777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacb350-ef61-43bc-925d-6ac31b37563e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.067064 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacb350-ef61-43bc-925d-6ac31b37563e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.067347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacb350-ef61-43bc-925d-6ac31b37563e None admin] PATH_INFO: `/v3/users/f39333e4f0d0425a99da37e75c007b5f` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.068716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacb350-ef61-43bc-925d-6ac31b37563e None admin] RBAC: Authorizing `identity:get_user(user_id=f39333e4f0d0425a99da37e75c007b5f)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.070202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacb350-ef61-43bc-925d-6ac31b37563e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.071987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 148/290] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:30:02 2022] GET /identity/v3/users/f39333e4f0d0425a99da37e75c007b5f => generated 297 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:03.079859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacb350-ef61-43bc-925d-6ac31b37563e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:03.111129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:03.126521 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 143/291] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:30:02 2022] PUT /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/users/ab407f2ac3c44e6285a7add6d54e806d/roles/d8a54178fbc44a49ad0df8d85a2adf0b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:03.233023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacb350-ef61-43bc-925d-6ac31b37563e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.291262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575dfb6-a828-4e99-8943-e2e2366a4260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.292257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575dfb6-a828-4e99-8943-e2e2366a4260 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.292758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575dfb6-a828-4e99-8943-e2e2366a4260 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.293281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575dfb6-a828-4e99-8943-e2e2366a4260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.393802 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575dfb6-a828-4e99-8943-e2e2366a4260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.395844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575dfb6-a828-4e99-8943-e2e2366a4260 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.430687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:03.557444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 149/292] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:03.564682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822472b6-1f6d-48dc-8276-80e2edf484d4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:03.614731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:03.682962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822472b6-1f6d-48dc-8276-80e2edf484d4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.722886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d27fb-108d-49a8-8e6a-d02e0b629a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.723859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d27fb-108d-49a8-8e6a-d02e0b629a5a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.724771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d27fb-108d-49a8-8e6a-d02e0b629a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.725081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d27fb-108d-49a8-8e6a-d02e0b629a5a None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.727067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d27fb-108d-49a8-8e6a-d02e0b629a5a None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.729584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d27fb-108d-49a8-8e6a-d02e0b629a5a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.732834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 144/293] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:03 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:03.742401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575dfb6-a828-4e99-8943-e2e2366a4260 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:03.868436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea55d-7cce-4a83-bb06-8aeecdffd8fa None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.868790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea55d-7cce-4a83-bb06-8aeecdffd8fa None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.868972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea55d-7cce-4a83-bb06-8aeecdffd8fa None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.870337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 145/294] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:03.874461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575dfb6-a828-4e99-8943-e2e2366a4260 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:03.881161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd04b70-1e67-4021-964a-825687f81163 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.881526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd04b70-1e67-4021-964a-825687f81163 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.881812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd04b70-1e67-4021-964a-825687f81163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.915645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-0bd04b70-1e67-4021-964a-825687f81163 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:03.917757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0bd04b70-1e67-4021-964a-825687f81163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:03.927904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:03.947971 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866283e-9bf0-419e-9a62-e7a8d0fff03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:03.948594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866283e-9bf0-419e-9a62-e7a8d0fff03c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.948986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866283e-9bf0-419e-9a62-e7a8d0fff03c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.949334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866283e-9bf0-419e-9a62-e7a8d0fff03c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.950274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866283e-9bf0-419e-9a62-e7a8d0fff03c None admin] RBAC: Adding query filter params (role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4, user.id=f39333e4f0d0425a99da37e75c007b5f) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:03.950636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866283e-9bf0-419e-9a62-e7a8d0fff03c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.952179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866283e-9bf0-419e-9a62-e7a8d0fff03c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.959316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 150/295] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:03 2022] GET /identity/v3/role_assignments?user.id=f39333e4f0d0425a99da37e75c007b5f&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:03.965005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa87aa4-5ebd-4510-8ebf-96da6a8fc372 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.965005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa87aa4-5ebd-4510-8ebf-96da6a8fc372 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.965005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa87aa4-5ebd-4510-8ebf-96da6a8fc372 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:03.965005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 151/296] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:03.970120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ad473-11f9-4989-9b83-ef81ad38f438 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:03.970458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ad473-11f9-4989-9b83-ef81ad38f438 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:03.975211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ad473-11f9-4989-9b83-ef81ad38f438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.026836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-319ad473-11f9-4989-9b83-ef81ad38f438 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.028048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-319ad473-11f9-4989-9b83-ef81ad38f438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.041775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 146/297] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:04.154864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 152/298] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:04.351793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd04b70-1e67-4021-964a-825687f81163 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.405169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:04.456761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c20fb3-0418-478f-86d7-fdc8caaa69ad None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.457088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c20fb3-0418-478f-86d7-fdc8caaa69ad None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.457355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c20fb3-0418-478f-86d7-fdc8caaa69ad None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.460146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 153/299] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:04.464450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc302b-e174-4c86-97f2-80c59ed577fe None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.464737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc302b-e174-4c86-97f2-80c59ed577fe None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.465016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc302b-e174-4c86-97f2-80c59ed577fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.494714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-45bc302b-e174-4c86-97f2-80c59ed577fe None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:04.496110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-45bc302b-e174-4c86-97f2-80c59ed577fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:04.499846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd04b70-1e67-4021-964a-825687f81163 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.545708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15c939-0480-4cda-b393-202b5206a4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.546348 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15c939-0480-4cda-b393-202b5206a4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.546660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15c939-0480-4cda-b393-202b5206a4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.547630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15c939-0480-4cda-b393-202b5206a4b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.549068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15c939-0480-4cda-b393-202b5206a4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.550580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15c939-0480-4cda-b393-202b5206a4b5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.552343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 147/300] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:04.553052 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15c939-0480-4cda-b393-202b5206a4b5 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.591545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 154/301] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:30:04.592908 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc302b-e174-4c86-97f2-80c59ed577fe None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.670336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:04.739884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15c939-0480-4cda-b393-202b5206a4b5 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.757326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:04.772830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc302b-e174-4c86-97f2-80c59ed577fe None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:04.790131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce283e7-3dad-40cc-8ff0-f5e8ef9d1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.790636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce283e7-3dad-40cc-8ff0-f5e8ef9d1594 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.790921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce283e7-3dad-40cc-8ff0-f5e8ef9d1594 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.791197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce283e7-3dad-40cc-8ff0-f5e8ef9d1594 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.804458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce283e7-3dad-40cc-8ff0-f5e8ef9d1594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-8ce283e7-3dad-40cc-8ff0-f5e8ef9d1594 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:04.809499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:04.811587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:04.813725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:04.817060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 148/302] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:04.817964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce283e7-3dad-40cc-8ff0-f5e8ef9d1594 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:04.834739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2d84f-52ce-42ef-92d2-34265952b0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:04.835887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d84f-52ce-42ef-92d2-34265952b0de None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:04.836372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d84f-52ce-42ef-92d2-34265952b0de None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:04.836911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d84f-52ce-42ef-92d2-34265952b0de None admin] PATH_INFO: `/v3/groups` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:04.837972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2d84f-52ce-42ef-92d2-34265952b0de None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:04.840412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2d84f-52ce-42ef-92d2-34265952b0de None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:04.961302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce283e7-3dad-40cc-8ff0-f5e8ef9d1594 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.015038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4c734-ce66-4c71-9737-6606d7fce189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.015778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4c734-ce66-4c71-9737-6606d7fce189 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.016202 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4c734-ce66-4c71-9737-6606d7fce189 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.016576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4c734-ce66-4c71-9737-6606d7fce189 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.017269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4c734-ce66-4c71-9737-6606d7fce189 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:05.017575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4c734-ce66-4c71-9737-6606d7fce189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.019331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4c734-ce66-4c71-9737-6606d7fce189 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.027994 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 149/303] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:05.029243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4c734-ce66-4c71-9737-6606d7fce189 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.063101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.142943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4c734-ce66-4c71-9737-6606d7fce189 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.179436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ed5f5-61d5-4edc-a6dd-bab4181c77d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.180069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ed5f5-61d5-4edc-a6dd-bab4181c77d9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.180643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ed5f5-61d5-4edc-a6dd-bab4181c77d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.180643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ed5f5-61d5-4edc-a6dd-bab4181c77d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.190597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ed5f5-61d5-4edc-a6dd-bab4181c77d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-b56ed5f5-61d5-4edc-a6dd-bab4181c77d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:05.195871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:05.197148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 150/304] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:05.198075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ed5f5-61d5-4edc-a6dd-bab4181c77d9 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.233322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 155/305] 10.222.0.82 () {60 vars in 1134 bytes} [Wed Aug 24 03:30:04 2022] POST /identity/v3/groups => generated 229 bytes in 641 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:05.236001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2d84f-52ce-42ef-92d2-34265952b0de None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.382887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.406953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ed5f5-61d5-4edc-a6dd-bab4181c77d9 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.434415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2d84f-52ce-42ef-92d2-34265952b0de None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.444456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.480380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c286b-6629-4e66-86c3-8e6f56165f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.481041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c286b-6629-4e66-86c3-8e6f56165f02 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.481325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c286b-6629-4e66-86c3-8e6f56165f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.481603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c286b-6629-4e66-86c3-8e6f56165f02 None admin] PATH_INFO: `/v3/users/50b302b8ab574cad8de1344e2c49a99a` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.495625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ea5f-c819-4fa8-99af-c24830b8d0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.495625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ea5f-c819-4fa8-99af-c24830b8d0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.495625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ea5f-c819-4fa8-99af-c24830b8d0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.495625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ea5f-c819-4fa8-99af-c24830b8d0d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.496280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12ea5f-c819-4fa8-99af-c24830b8d0d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:05.496280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12ea5f-c819-4fa8-99af-c24830b8d0d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.496280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12ea5f-c819-4fa8-99af-c24830b8d0d1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.498809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c286b-6629-4e66-86c3-8e6f56165f02 None admin] RBAC: Authorizing `identity:get_user(user_id=50b302b8ab574cad8de1344e2c49a99a)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.500830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c286b-6629-4e66-86c3-8e6f56165f02 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.503560 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 156/307] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:05.505412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 151/307] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/users/50b302b8ab574cad8de1344e2c49a99a => generated 293 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:05.511681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ea5f-c819-4fa8-99af-c24830b8d0d1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.513213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c286b-6629-4e66-86c3-8e6f56165f02 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.662460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ea5f-c819-4fa8-99af-c24830b8d0d1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.695521 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c286b-6629-4e66-86c3-8e6f56165f02 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.718543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e92bd-391a-4dae-83db-820f94c23354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.720494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e92bd-391a-4dae-83db-820f94c23354 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.721241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e92bd-391a-4dae-83db-820f94c23354 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.721526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e92bd-391a-4dae-83db-820f94c23354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.722711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e92bd-391a-4dae-83db-820f94c23354 None admin] RBAC: Adding query filter params (role.id=d8a54178fbc44a49ad0df8d85a2adf0b, user.id=50b302b8ab574cad8de1344e2c49a99a, scope.system=all) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:05.723030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e92bd-391a-4dae-83db-820f94c23354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.724538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e92bd-391a-4dae-83db-820f94c23354 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.742180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 157/308] 10.222.0.82 () {56 vars in 1308 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/role_assignments?user.id=50b302b8ab574cad8de1344e2c49a99a&scope.system=all&role.id=d8a54178fbc44a49ad0df8d85a2adf0b => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:05.752758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be3d73-5054-419e-a715-5c8cbb470726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.753420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be3d73-5054-419e-a715-5c8cbb470726 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.758540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be3d73-5054-419e-a715-5c8cbb470726 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.758540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be3d73-5054-419e-a715-5c8cbb470726 None admin] PATH_INFO: `/v3/users/3d89c259abf34407bf5daab2a3f1fb47` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.758540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be3d73-5054-419e-a715-5c8cbb470726 None admin] RBAC: Authorizing `identity:get_user(user_id=3d89c259abf34407bf5daab2a3f1fb47)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.758540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be3d73-5054-419e-a715-5c8cbb470726 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.759307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 152/309] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/users/3d89c259abf34407bf5daab2a3f1fb47 => generated 293 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:05.764748 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e92bd-391a-4dae-83db-820f94c23354 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.803206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.903104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6bfa-6d9b-4b7f-a0a4-48e1fb706563 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.903495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6bfa-6d9b-4b7f-a0a4-48e1fb706563 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.903898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6bfa-6d9b-4b7f-a0a4-48e1fb706563 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.911759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 153/310] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:05.913204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e584bd5-395c-4fa5-a893-34030fabf09c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.913492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e584bd5-395c-4fa5-a893-34030fabf09c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.913774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e584bd5-395c-4fa5-a893-34030fabf09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.927999 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e92bd-391a-4dae-83db-820f94c23354 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.940488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-0e584bd5-395c-4fa5-a893-34030fabf09c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:05.941757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0e584bd5-395c-4fa5-a893-34030fabf09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:05.952704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.973169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd93c0-1f3b-4b28-9891-f9b9e3961c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.974001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd93c0-1f3b-4b28-9891-f9b9e3961c40 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.974321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd93c0-1f3b-4b28-9891-f9b9e3961c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.975426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd93c0-1f3b-4b28-9891-f9b9e3961c40 None admin] PATH_INFO: `/v3/system/users/3d89c259abf34407bf5daab2a3f1fb47/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.978116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd93c0-1f3b-4b28-9891-f9b9e3961c40 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3d89c259abf34407bf5daab2a3f1fb47, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.978537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd93c0-1f3b-4b28-9891-f9b9e3961c40 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:06.058596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 154/311] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:30:06.101400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 158/312] 10.222.0.82 () {58 vars in 1334 bytes} [Wed Aug 24 03:30:05 2022] PUT /identity/v3/system/users/3d89c259abf34407bf5daab2a3f1fb47/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:06.385678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e584bd5-395c-4fa5-a893-34030fabf09c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:06.516969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e584bd5-395c-4fa5-a893-34030fabf09c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:06.558589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c6e4a-a0f0-40b6-9ea5-fc5fe09833e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:06.559531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c6e4a-a0f0-40b6-9ea5-fc5fe09833e8 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.559962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c6e4a-a0f0-40b6-9ea5-fc5fe09833e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.560395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c6e4a-a0f0-40b6-9ea5-fc5fe09833e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.573165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c6e4a-a0f0-40b6-9ea5-fc5fe09833e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-4f6c6e4a-a0f0-40b6-9ea5-fc5fe09833e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:06.579721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:06.580825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:06.581753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:06.584045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 155/313] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:06.589909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd93c0-1f3b-4b28-9891-f9b9e3961c40 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:06.710874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:06.716763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd93c0-1f3b-4b28-9891-f9b9e3961c40 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:06.769460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad3098-738a-4fb4-99fc-63fd682d43a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:06.770417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad3098-738a-4fb4-99fc-63fd682d43a1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:06.770884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad3098-738a-4fb4-99fc-63fd682d43a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:06.771327 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad3098-738a-4fb4-99fc-63fd682d43a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:06.772378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad3098-738a-4fb4-99fc-63fd682d43a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:06.772868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad3098-738a-4fb4-99fc-63fd682d43a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:06.775291 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad3098-738a-4fb4-99fc-63fd682d43a1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:06.791222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 159/314] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:06.801248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c6e4a-a0f0-40b6-9ea5-fc5fe09833e8 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:06.808146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:06.964414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c6e4a-a0f0-40b6-9ea5-fc5fe09833e8 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.023626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f51d8-a909-4fda-b9c7-4cb14bcf1267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.024301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f51d8-a909-4fda-b9c7-4cb14bcf1267 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.024585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f51d8-a909-4fda-b9c7-4cb14bcf1267 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.024865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f51d8-a909-4fda-b9c7-4cb14bcf1267 None admin] PATH_INFO: `/v3/users/ab407f2ac3c44e6285a7add6d54e806d` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.026048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f51d8-a909-4fda-b9c7-4cb14bcf1267 None admin] RBAC: Authorizing `identity:get_user(user_id=ab407f2ac3c44e6285a7add6d54e806d)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.027546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f51d8-a909-4fda-b9c7-4cb14bcf1267 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.030498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 156/315] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:30:06 2022] GET /identity/v3/users/ab407f2ac3c44e6285a7add6d54e806d => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:07.037274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad3098-738a-4fb4-99fc-63fd682d43a1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.165068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad3098-738a-4fb4-99fc-63fd682d43a1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.204106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a125d-c8d3-4be6-9cae-e377ff76f325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.205213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a125d-c8d3-4be6-9cae-e377ff76f325 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.205504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a125d-c8d3-4be6-9cae-e377ff76f325 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.205791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a125d-c8d3-4be6-9cae-e377ff76f325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.234344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:07.286369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a125d-c8d3-4be6-9cae-e377ff76f325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.287911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a125d-c8d3-4be6-9cae-e377ff76f325 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.437935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 160/316] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:07.444587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f51d8-a909-4fda-b9c7-4cb14bcf1267 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.460212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bc5ce-5d3c-4cfb-a6ef-0ab60cac421b None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.461157 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bc5ce-5d3c-4cfb-a6ef-0ab60cac421b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.461528 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bc5ce-5d3c-4cfb-a6ef-0ab60cac421b None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.464546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 161/317] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:07.471883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bde25-35cb-4fd5-9c39-11f8eb618723 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.472184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bde25-35cb-4fd5-9c39-11f8eb618723 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.472463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bde25-35cb-4fd5-9c39-11f8eb618723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.492078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-cf0bde25-35cb-4fd5-9c39-11f8eb618723 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:07.493190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cf0bde25-35cb-4fd5-9c39-11f8eb618723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:07.543715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:07.552829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:07.582677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 162/318] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:07.587027 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f51d8-a909-4fda-b9c7-4cb14bcf1267 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.633330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99ed90-af4a-4b6f-abb5-bdc2081a139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.633982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99ed90-af4a-4b6f-abb5-bdc2081a139e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.634266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99ed90-af4a-4b6f-abb5-bdc2081a139e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.634562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99ed90-af4a-4b6f-abb5-bdc2081a139e None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.635795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99ed90-af4a-4b6f-abb5-bdc2081a139e None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.637283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99ed90-af4a-4b6f-abb5-bdc2081a139e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.639320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 157/319] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:07 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:07.646660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bde25-35cb-4fd5-9c39-11f8eb618723 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.777190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bde25-35cb-4fd5-9c39-11f8eb618723 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:07.817727 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f347041-25fc-4362-996d-9d52146d135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:07.818359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f347041-25fc-4362-996d-9d52146d135b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.818652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f347041-25fc-4362-996d-9d52146d135b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.818939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f347041-25fc-4362-996d-9d52146d135b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.819696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f347041-25fc-4362-996d-9d52146d135b None admin] RBAC: Adding query filter params (role.id=d8a54178fbc44a49ad0df8d85a2adf0b, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4, user.id=ab407f2ac3c44e6285a7add6d54e806d) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:07.820009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f347041-25fc-4362-996d-9d52146d135b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:07.821483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f347041-25fc-4362-996d-9d52146d135b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:07.828709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 163/320] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:07 2022] GET /identity/v3/role_assignments?user.id=ab407f2ac3c44e6285a7add6d54e806d&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=d8a54178fbc44a49ad0df8d85a2adf0b => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:07.886165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99ed90-af4a-4b6f-abb5-bdc2081a139e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:07.982782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc048d7f-aedb-4161-b2f9-1d2a73f8889a None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.983087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc048d7f-aedb-4161-b2f9-1d2a73f8889a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.983346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc048d7f-aedb-4161-b2f9-1d2a73f8889a None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:07.985700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 164/321] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:07.991465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0976f3-3e0b-4b26-8a31-2ca38d2c311e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:07.991823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0976f3-3e0b-4b26-8a31-2ca38d2c311e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:07.992154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0976f3-3e0b-4b26-8a31-2ca38d2c311e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.022937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-ca0976f3-3e0b-4b26-8a31-2ca38d2c311e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:08.024654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ca0976f3-3e0b-4b26-8a31-2ca38d2c311e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:08.032863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99ed90-af4a-4b6f-abb5-bdc2081a139e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.093323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:08.094380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbbd6b-04c1-4ecd-afc0-ca8980f4c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.096623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbbd6b-04c1-4ecd-afc0-ca8980f4c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.096623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbbd6b-04c1-4ecd-afc0-ca8980f4c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.096623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbbd6b-04c1-4ecd-afc0-ca8980f4c2d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.097648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbbd6b-04c1-4ecd-afc0-ca8980f4c2d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.099483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbbd6b-04c1-4ecd-afc0-ca8980f4c2d7 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.101987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 158/322] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:08.108339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbbd6b-04c1-4ecd-afc0-ca8980f4c2d7 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.121991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:08.129923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 165/323] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:30:08.244409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbbd6b-04c1-4ecd-afc0-ca8980f4c2d7 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.311484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43762649-d861-43e5-830a-d6abde008506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.312403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43762649-d861-43e5-830a-d6abde008506 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.312867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43762649-d861-43e5-830a-d6abde008506 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.313341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43762649-d861-43e5-830a-d6abde008506 None admin] PATH_INFO: `/v3/groups` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.314426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43762649-d861-43e5-830a-d6abde008506 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.316616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43762649-d861-43e5-830a-d6abde008506 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.404685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0976f3-3e0b-4b26-8a31-2ca38d2c311e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.452788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 159/324] 10.222.0.82 () {60 vars in 1134 bytes} [Wed Aug 24 03:30:08 2022] POST /identity/v3/groups => generated 226 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:08.465015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db381e7a-a056-45c0-819a-c973a34e3774 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.465365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db381e7a-a056-45c0-819a-c973a34e3774 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.465577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db381e7a-a056-45c0-819a-c973a34e3774 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.467296 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 160/325] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:08.474990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44156835-865e-4d02-b22a-9c48b3480ada None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.475580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44156835-865e-4d02-b22a-9c48b3480ada None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.476001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44156835-865e-4d02-b22a-9c48b3480ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.504287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-44156835-865e-4d02-b22a-9c48b3480ada None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:08.506377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-44156835-865e-4d02-b22a-9c48b3480ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:08.545436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0976f3-3e0b-4b26-8a31-2ca38d2c311e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.591721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27346f-0a67-46c8-85ac-f57b5b33076e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.592546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27346f-0a67-46c8-85ac-f57b5b33076e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.592666 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:08.592941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27346f-0a67-46c8-85ac-f57b5b33076e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.593329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27346f-0a67-46c8-85ac-f57b5b33076e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.600275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 161/326] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:08.615684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27346f-0a67-46c8-85ac-f57b5b33076e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-cb27346f-0a67-46c8-85ac-f57b5b33076e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:08.620737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:08.621667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:08.623110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 166/327] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:08.629622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44156835-865e-4d02-b22a-9c48b3480ada None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.771608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44156835-865e-4d02-b22a-9c48b3480ada None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.826016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020ec05-e5b8-4c73-994f-f443da90bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.826697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020ec05-e5b8-4c73-994f-f443da90bae1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.826984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020ec05-e5b8-4c73-994f-f443da90bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.827258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020ec05-e5b8-4c73-994f-f443da90bae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.827939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020ec05-e5b8-4c73-994f-f443da90bae1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:08.828823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020ec05-e5b8-4c73-994f-f443da90bae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.830313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020ec05-e5b8-4c73-994f-f443da90bae1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.840303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 162/328] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:08.846289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27346f-0a67-46c8-85ac-f57b5b33076e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.911458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:08.924048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020ec05-e5b8-4c73-994f-f443da90bae1 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:09.018498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27346f-0a67-46c8-85ac-f57b5b33076e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:09.047484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:09.090761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdd545-6c5c-470d-a5b6-52bf98409260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:09.091406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdd545-6c5c-470d-a5b6-52bf98409260 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.091684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdd545-6c5c-470d-a5b6-52bf98409260 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.092002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdd545-6c5c-470d-a5b6-52bf98409260 None admin] PATH_INFO: `/v3/users/50b302b8ab574cad8de1344e2c49a99a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.093435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bdd545-6c5c-470d-a5b6-52bf98409260 None admin] RBAC: Authorizing `identity:get_user(user_id=50b302b8ab574cad8de1344e2c49a99a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:09.095629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bdd545-6c5c-470d-a5b6-52bf98409260 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:09.098379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 167/329] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:30:08 2022] GET /identity/v3/users/50b302b8ab574cad8de1344e2c49a99a => generated 293 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:09.107569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020ec05-e5b8-4c73-994f-f443da90bae1 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:09.107910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdd545-6c5c-470d-a5b6-52bf98409260 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:09.147909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e640a-0fd1-4c25-b9c8-b2f30a134fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:09.148565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e640a-0fd1-4c25-b9c8-b2f30a134fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.148894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e640a-0fd1-4c25-b9c8-b2f30a134fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.149175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e640a-0fd1-4c25-b9c8-b2f30a134fa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.169552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e640a-0fd1-4c25-b9c8-b2f30a134fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-1a4e640a-0fd1-4c25-b9c8-b2f30a134fa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:09.176988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:09.177961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:09.179632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 163/330] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:09.185385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e640a-0fd1-4c25-b9c8-b2f30a134fa1 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:09.274497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdd545-6c5c-470d-a5b6-52bf98409260 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:09.290350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:09.329877 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19ba87-1708-4522-add1-73957b96759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:09.330604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19ba87-1708-4522-add1-73957b96759c None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.330895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19ba87-1708-4522-add1-73957b96759c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.331177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19ba87-1708-4522-add1-73957b96759c None admin] PATH_INFO: `/v3/system/users/50b302b8ab574cad8de1344e2c49a99a/roles/d8a54178fbc44a49ad0df8d85a2adf0b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.332788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e19ba87-1708-4522-add1-73957b96759c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=50b302b8ab574cad8de1344e2c49a99a, role_id=d8a54178fbc44a49ad0df8d85a2adf0b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:09.334393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e19ba87-1708-4522-add1-73957b96759c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:09.348843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e640a-0fd1-4c25-b9c8-b2f30a134fa1 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:09.373238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:09.405340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840899c4-d3a6-4f24-84b5-8c0502a275b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:09.405966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840899c4-d3a6-4f24-84b5-8c0502a275b5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.406907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840899c4-d3a6-4f24-84b5-8c0502a275b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.407185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840899c4-d3a6-4f24-84b5-8c0502a275b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.407924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840899c4-d3a6-4f24-84b5-8c0502a275b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:09.408234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840899c4-d3a6-4f24-84b5-8c0502a275b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:09.409700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840899c4-d3a6-4f24-84b5-8c0502a275b5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:09.417981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 168/331] 10.222.0.82 () {58 vars in 1334 bytes} [Wed Aug 24 03:30:09 2022] PUT /identity/v3/system/users/50b302b8ab574cad8de1344e2c49a99a/roles/d8a54178fbc44a49ad0df8d85a2adf0b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:09.420753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 164/332] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:09.428139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19ba87-1708-4522-add1-73957b96759c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:09.581202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19ba87-1708-4522-add1-73957b96759c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:09.630088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7fac8-e62c-4412-8a72-01c5a98c1a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:09.631092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7fac8-e62c-4412-8a72-01c5a98c1a7d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.631626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7fac8-e62c-4412-8a72-01c5a98c1a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.632088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7fac8-e62c-4412-8a72-01c5a98c1a7d None admin] PATH_INFO: `/v3/users/3d89c259abf34407bf5daab2a3f1fb47` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.633550 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7fac8-e62c-4412-8a72-01c5a98c1a7d None admin] RBAC: Authorizing `identity:get_user(user_id=3d89c259abf34407bf5daab2a3f1fb47)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:09.635232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7fac8-e62c-4412-8a72-01c5a98c1a7d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:09.637390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 169/333] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:30:09 2022] GET /identity/v3/users/3d89c259abf34407bf5daab2a3f1fb47 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:09.645621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840899c4-d3a6-4f24-84b5-8c0502a275b5 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:09.774885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840899c4-d3a6-4f24-84b5-8c0502a275b5 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:09.826273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6a719-a5a7-4af9-8822-4d0bd5a07977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:09.827184 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6a719-a5a7-4af9-8822-4d0bd5a07977 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:09.827603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6a719-a5a7-4af9-8822-4d0bd5a07977 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:09.828034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6a719-a5a7-4af9-8822-4d0bd5a07977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:09.829688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6a719-a5a7-4af9-8822-4d0bd5a07977 None admin] RBAC: Adding query filter params (role.id=5bf6686ae6624e84aba335589d8a98a4, user.id=3d89c259abf34407bf5daab2a3f1fb47, scope.system=all) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:09.830189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6a719-a5a7-4af9-8822-4d0bd5a07977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:09.832515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6a719-a5a7-4af9-8822-4d0bd5a07977 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:09.853294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 165/334] 10.222.0.82 () {56 vars in 1308 bytes} [Wed Aug 24 03:30:09 2022] GET /identity/v3/role_assignments?user.id=3d89c259abf34407bf5daab2a3f1fb47&scope.system=all&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:10.874564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5502cf-126a-400c-9a0a-ed398ce06247 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:10.874564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5502cf-126a-400c-9a0a-ed398ce06247 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:10.874564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5502cf-126a-400c-9a0a-ed398ce06247 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:10.876175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 170/335] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:10.881667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc8b49-4c5d-43d3-a1ad-6e2361d110de None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:10.882120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc8b49-4c5d-43d3-a1ad-6e2361d110de None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:10.882225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc8b49-4c5d-43d3-a1ad-6e2361d110de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:10.905775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-75bc8b49-4c5d-43d3-a1ad-6e2361d110de None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:10.908181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-75bc8b49-4c5d-43d3-a1ad-6e2361d110de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:10.952597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:10.994613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 166/336] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:11.301794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5502cf-126a-400c-9a0a-ed398ce06247 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:11.358210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:11.432406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5502cf-126a-400c-9a0a-ed398ce06247 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:11.474046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1346ff-a669-4555-95c8-4babe3e10eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:11.474742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1346ff-a669-4555-95c8-4babe3e10eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.475027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1346ff-a669-4555-95c8-4babe3e10eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.475300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1346ff-a669-4555-95c8-4babe3e10eb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:11.490083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1346ff-a669-4555-95c8-4babe3e10eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-bf1346ff-a669-4555-95c8-4babe3e10eb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:11.495305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:11.496256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:11.497090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:11.499110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 171/337] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:11.499627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705b1fb-0cff-441d-b2f2-a772fd3be6c3 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.500025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705b1fb-0cff-441d-b2f2-a772fd3be6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.500771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705b1fb-0cff-441d-b2f2-a772fd3be6c3 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:11.502228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 167/338] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:11.502912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1346ff-a669-4555-95c8-4babe3e10eb7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:11.521097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f21987-e1e7-4815-a88d-f7ea1703cdba None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.521396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f21987-e1e7-4815-a88d-f7ea1703cdba None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.521647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f21987-e1e7-4815-a88d-f7ea1703cdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:11.567668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d6f21987-e1e7-4815-a88d-f7ea1703cdba None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:11.568798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d6f21987-e1e7-4815-a88d-f7ea1703cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:11.671367 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:11.676071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1346ff-a669-4555-95c8-4babe3e10eb7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:11.679667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 168/339] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:30:11.729536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf64a8-b5df-473d-be3d-599a80af0b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:11.730681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf64a8-b5df-473d-be3d-599a80af0b80 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.731194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf64a8-b5df-473d-be3d-599a80af0b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.731648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf64a8-b5df-473d-be3d-599a80af0b80 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:11.732531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf64a8-b5df-473d-be3d-599a80af0b80 None admin] RBAC: Adding query filter params (name=member) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:11.733029 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf64a8-b5df-473d-be3d-599a80af0b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:11.735037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf64a8-b5df-473d-be3d-599a80af0b80 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:11.750809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 172/340] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:11.757069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f21987-e1e7-4815-a88d-f7ea1703cdba None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:11.893997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:11.900628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f21987-e1e7-4815-a88d-f7ea1703cdba None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:11.941463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdd45b-8d61-4633-9499-de093eaa48c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:11.942105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdd45b-8d61-4633-9499-de093eaa48c7 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.942386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdd45b-8d61-4633-9499-de093eaa48c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.942686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdd45b-8d61-4633-9499-de093eaa48c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.043531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdd45b-8d61-4633-9499-de093eaa48c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:12.045020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdd45b-8d61-4633-9499-de093eaa48c7 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:12.129709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf64a8-b5df-473d-be3d-599a80af0b80 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:12.179592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:12.188991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 169/341] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:12.209409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdd45b-8d61-4633-9499-de093eaa48c7 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:12.287111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf64a8-b5df-473d-be3d-599a80af0b80 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:12.327465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d59d9-4ba0-4d64-9be7-136fe6a80ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:12.328517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d59d9-4ba0-4d64-9be7-136fe6a80ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.329099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d59d9-4ba0-4d64-9be7-136fe6a80ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.329677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d59d9-4ba0-4d64-9be7-136fe6a80ee3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.341330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d59d9-4ba0-4d64-9be7-136fe6a80ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-302d59d9-4ba0-4d64-9be7-136fe6a80ee3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:12.347555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:12.348467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:12.349288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:12.351642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 173/342] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:30:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:12.353195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1edd5-6831-4a14-a437-3c7605206621 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.353686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1edd5-6831-4a14-a437-3c7605206621 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.354177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1edd5-6831-4a14-a437-3c7605206621 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.355731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 174/343] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:12.357666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1edd5-6831-4a14-a437-3c7605206621 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:12.387784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:12.436804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdd45b-8d61-4633-9499-de093eaa48c7 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:12.477412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1edd5-6831-4a14-a437-3c7605206621 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:12.503943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2d2cf-b18e-4c7a-94c3-8f20bd87b6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:12.504601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d2cf-b18e-4c7a-94c3-8f20bd87b6be None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.504894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d2cf-b18e-4c7a-94c3-8f20bd87b6be None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.505185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2d2cf-b18e-4c7a-94c3-8f20bd87b6be None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.511319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2d2cf-b18e-4c7a-94c3-8f20bd87b6be None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:12.512818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2d2cf-b18e-4c7a-94c3-8f20bd87b6be None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:12.514554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 170/344] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:12 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:12.517956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3308a-7e6e-43d1-8bb9-9d7f38b07a7f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.517956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3308a-7e6e-43d1-8bb9-9d7f38b07a7f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.517956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3308a-7e6e-43d1-8bb9-9d7f38b07a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.527298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d789d-b9fd-460a-a630-cc3cb9b0fcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:12.528785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d789d-b9fd-460a-a630-cc3cb9b0fcba None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.529610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d789d-b9fd-460a-a630-cc3cb9b0fcba None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.530348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d789d-b9fd-460a-a630-cc3cb9b0fcba None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.532764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d789d-b9fd-460a-a630-cc3cb9b0fcba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:12.533692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d789d-b9fd-460a-a630-cc3cb9b0fcba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:12.536588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d789d-b9fd-460a-a630-cc3cb9b0fcba None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:12.554828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 175/345] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:12.556985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d789d-b9fd-460a-a630-cc3cb9b0fcba None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:12.559535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9bf3308a-7e6e-43d1-8bb9-9d7f38b07a7f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:12.560770 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9bf3308a-7e6e-43d1-8bb9-9d7f38b07a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:12.665228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:12.678154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 171/346] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:12.679721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3308a-7e6e-43d1-8bb9-9d7f38b07a7f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:12.771989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:12.807726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d789d-b9fd-460a-a630-cc3cb9b0fcba None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:12.884030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3308a-7e6e-43d1-8bb9-9d7f38b07a7f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:12.886643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad1ec7-9128-4428-967b-c07ef79a7785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:12.887585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad1ec7-9128-4428-967b-c07ef79a7785 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.888074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad1ec7-9128-4428-967b-c07ef79a7785 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.888552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad1ec7-9128-4428-967b-c07ef79a7785 None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.901593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad1ec7-9128-4428-967b-c07ef79a7785 None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:12.903436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad1ec7-9128-4428-967b-c07ef79a7785 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:12.905442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 176/347] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:12 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:12.907678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af5f94-3c86-44a1-9d70-83a972cfafcf None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.908201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af5f94-3c86-44a1-9d70-83a972cfafcf None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.908667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af5f94-3c86-44a1-9d70-83a972cfafcf None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:12.914822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 177/348] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:12.915915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af5f94-3c86-44a1-9d70-83a972cfafcf None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:12.940228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e553c4-1d6c-4fc6-a0fa-77486e9ececf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:12.940919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e553c4-1d6c-4fc6-a0fa-77486e9ececf None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:12.941147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e553c4-1d6c-4fc6-a0fa-77486e9ececf None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:12.941426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e553c4-1d6c-4fc6-a0fa-77486e9ececf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.044092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:13.049575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e553c4-1d6c-4fc6-a0fa-77486e9ececf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:13.051413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e553c4-1d6c-4fc6-a0fa-77486e9ececf None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:13.089250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af5f94-3c86-44a1-9d70-83a972cfafcf None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:13.151381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e01634-3784-4ac8-8d2c-56a8b9f74733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:13.152924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e01634-3784-4ac8-8d2c-56a8b9f74733 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.153380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e01634-3784-4ac8-8d2c-56a8b9f74733 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.154068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e01634-3784-4ac8-8d2c-56a8b9f74733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.155488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e01634-3784-4ac8-8d2c-56a8b9f74733 None admin] RBAC: Adding query filter params (group.id=52609d57fd324f898cf5125b6e45e300, role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=86e412e829cf47228db2532cb4c7c4e4) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:13.155963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e01634-3784-4ac8-8d2c-56a8b9f74733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:13.158883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e01634-3784-4ac8-8d2c-56a8b9f74733 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:13.166463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 178/349] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:12 2022] GET /identity/v3/role_assignments?group.id=52609d57fd324f898cf5125b6e45e300&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:13.168594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3056cc-4f44-4bb6-a26f-19389cd84d7a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.169054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3056cc-4f44-4bb6-a26f-19389cd84d7a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.169464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3056cc-4f44-4bb6-a26f-19389cd84d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.196196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-0d3056cc-4f44-4bb6-a26f-19389cd84d7a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:13.197705 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0d3056cc-4f44-4bb6-a26f-19389cd84d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:13.198007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 172/350] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:13.199037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e553c4-1d6c-4fc6-a0fa-77486e9ececf None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:13.207505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:13.341184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 179/351] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:13.342337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3056cc-4f44-4bb6-a26f-19389cd84d7a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:13.403730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:13.411324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e553c4-1d6c-4fc6-a0fa-77486e9ececf None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:13.470598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f023c33-19cf-43b1-9bff-91b77bc69d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:13.471255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f023c33-19cf-43b1-9bff-91b77bc69d6b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.471539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f023c33-19cf-43b1-9bff-91b77bc69d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.471816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f023c33-19cf-43b1-9bff-91b77bc69d6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.483532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f023c33-19cf-43b1-9bff-91b77bc69d6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-0f023c33-19cf-43b1-9bff-91b77bc69d6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:13.489635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:13.490630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:30:13.491458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:13.492859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 173/352] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:13.494146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb4859-fa25-4fdf-b066-9cd108abb5dc None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.494594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb4859-fa25-4fdf-b066-9cd108abb5dc None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.494758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb4859-fa25-4fdf-b066-9cd108abb5dc None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.496212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 174/353] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:13.498226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb4859-fa25-4fdf-b066-9cd108abb5dc None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:13.566164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3056cc-4f44-4bb6-a26f-19389cd84d7a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:13.617303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25058ae5-2af6-46ed-ae8b-13bda8faca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:13.617876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25058ae5-2af6-46ed-ae8b-13bda8faca8e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.618163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25058ae5-2af6-46ed-ae8b-13bda8faca8e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.618444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25058ae5-2af6-46ed-ae8b-13bda8faca8e None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.619662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25058ae5-2af6-46ed-ae8b-13bda8faca8e None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:13.621867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25058ae5-2af6-46ed-ae8b-13bda8faca8e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:13.624385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 180/354] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:13 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:13.627377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325402-a71c-43e2-99be-d17d3fa4569e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.627661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325402-a71c-43e2-99be-d17d3fa4569e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.627944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325402-a71c-43e2-99be-d17d3fa4569e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.651710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-04325402-a71c-43e2-99be-d17d3fa4569e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:13.653552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-04325402-a71c-43e2-99be-d17d3fa4569e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:13.659356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:13.680279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb4859-fa25-4fdf-b066-9cd108abb5dc None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:13.739247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 181/355] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:13.740129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04325402-a71c-43e2-99be-d17d3fa4569e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:13.763899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21724c-aa1c-4b4e-83ec-80db223105aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:13.764982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21724c-aa1c-4b4e-83ec-80db223105aa None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.765533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21724c-aa1c-4b4e-83ec-80db223105aa None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.766087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21724c-aa1c-4b4e-83ec-80db223105aa None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.767060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21724c-aa1c-4b4e-83ec-80db223105aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:13.767604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21724c-aa1c-4b4e-83ec-80db223105aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:13.769288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21724c-aa1c-4b4e-83ec-80db223105aa None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:13.793100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 175/356] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:13.798969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21724c-aa1c-4b4e-83ec-80db223105aa None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:13.847436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:13.871876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04325402-a71c-43e2-99be-d17d3fa4569e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:13.926633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009290f9-2820-4438-9ad6-af3f14be1cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:13.927299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009290f9-2820-4438-9ad6-af3f14be1cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.927580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009290f9-2820-4438-9ad6-af3f14be1cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.927852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009290f9-2820-4438-9ad6-af3f14be1cb3 None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.928980 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009290f9-2820-4438-9ad6-af3f14be1cb3 None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:13.930734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009290f9-2820-4438-9ad6-af3f14be1cb3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:13.933429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 182/357] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:13 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:13.935114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ad444-ace5-4b9f-aaa1-ebd92df788cd None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.935462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ad444-ace5-4b9f-aaa1-ebd92df788cd None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.936646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ad444-ace5-4b9f-aaa1-ebd92df788cd None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.938854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 183/358] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:13.944708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41caf7f-7176-4ca4-b143-ea84a9d51048 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.949506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41caf7f-7176-4ca4-b143-ea84a9d51048 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.949506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41caf7f-7176-4ca4-b143-ea84a9d51048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.969365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c41caf7f-7176-4ca4-b143-ea84a9d51048 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:13.972338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c41caf7f-7176-4ca4-b143-ea84a9d51048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:13.978953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21724c-aa1c-4b4e-83ec-80db223105aa None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:13.985555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.027593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487331ed-6c6e-4ac3-b1e5-d05376052a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.028606 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487331ed-6c6e-4ac3-b1e5-d05376052a5c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.029113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487331ed-6c6e-4ac3-b1e5-d05376052a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.029605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487331ed-6c6e-4ac3-b1e5-d05376052a5c None admin] PATH_INFO: `/v3/users/50b302b8ab574cad8de1344e2c49a99a` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.031620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487331ed-6c6e-4ac3-b1e5-d05376052a5c None admin] RBAC: Authorizing `identity:get_user(user_id=50b302b8ab574cad8de1344e2c49a99a)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.033673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487331ed-6c6e-4ac3-b1e5-d05376052a5c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.037005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 176/359] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:30:13 2022] GET /identity/v3/users/50b302b8ab574cad8de1344e2c49a99a => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:14.038046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487331ed-6c6e-4ac3-b1e5-d05376052a5c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.065578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 184/360] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:14.067092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41caf7f-7176-4ca4-b143-ea84a9d51048 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.161587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.178643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487331ed-6c6e-4ac3-b1e5-d05376052a5c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.194362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41caf7f-7176-4ca4-b143-ea84a9d51048 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.223037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4589a9-6b83-4e14-abad-ae2c2bb9086d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.223897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4589a9-6b83-4e14-abad-ae2c2bb9086d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.224181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4589a9-6b83-4e14-abad-ae2c2bb9086d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.224452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4589a9-6b83-4e14-abad-ae2c2bb9086d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.225645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4589a9-6b83-4e14-abad-ae2c2bb9086d None admin] RBAC: Adding query filter params (group.id=52609d57fd324f898cf5125b6e45e300, role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=86e412e829cf47228db2532cb4c7c4e4) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:14.225961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4589a9-6b83-4e14-abad-ae2c2bb9086d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.227684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4589a9-6b83-4e14-abad-ae2c2bb9086d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.236460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 177/361] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/role_assignments?group.id=52609d57fd324f898cf5125b6e45e300&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:14.237294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4589a9-6b83-4e14-abad-ae2c2bb9086d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.239115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4917d5e-ccc7-424d-9296-b5389a574d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.240079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4917d5e-ccc7-424d-9296-b5389a574d2f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.240569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4917d5e-ccc7-424d-9296-b5389a574d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.241042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4917d5e-ccc7-424d-9296-b5389a574d2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.251069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.256107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4917d5e-ccc7-424d-9296-b5389a574d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-f4917d5e-ccc7-424d-9296-b5389a574d2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:14.261776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:14.262791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:14.263835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:14.266363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 185/362] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:14.267518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4917d5e-ccc7-424d-9296-b5389a574d2f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.368578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4589a9-6b83-4e14-abad-ae2c2bb9086d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.438125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4917d5e-ccc7-424d-9296-b5389a574d2f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.443328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f033ab-899d-4585-8fee-2545a292ccdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.444087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f033ab-899d-4585-8fee-2545a292ccdf None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.444482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f033ab-899d-4585-8fee-2545a292ccdf None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.444865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f033ab-899d-4585-8fee-2545a292ccdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.446046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f033ab-899d-4585-8fee-2545a292ccdf None admin] RBAC: Adding query filter params (role.id=d8a54178fbc44a49ad0df8d85a2adf0b, user.id=50b302b8ab574cad8de1344e2c49a99a, scope.system=all) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:14.446472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f033ab-899d-4585-8fee-2545a292ccdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.448116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f033ab-899d-4585-8fee-2545a292ccdf None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.471493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 178/363] 10.222.0.82 () {56 vars in 1308 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/role_assignments?user.id=50b302b8ab574cad8de1344e2c49a99a&scope.system=all&role.id=d8a54178fbc44a49ad0df8d85a2adf0b => generated 510 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:14.472385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f033ab-899d-4585-8fee-2545a292ccdf None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.487388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef2586-e988-41fc-a511-5d43762bc148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.488384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef2586-e988-41fc-a511-5d43762bc148 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.488816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef2586-e988-41fc-a511-5d43762bc148 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.489314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef2586-e988-41fc-a511-5d43762bc148 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.493925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.503989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef2586-e988-41fc-a511-5d43762bc148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-3fef2586-e988-41fc-a511-5d43762bc148 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:14.509562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:14.510487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:14.511336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:14.514128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 186/364] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:14.521428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef2586-e988-41fc-a511-5d43762bc148 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.607997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.669532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef2586-e988-41fc-a511-5d43762bc148 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.673868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f033ab-899d-4585-8fee-2545a292ccdf None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.725816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c55bd-8caa-4331-904b-b32e883e1b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.726364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c55bd-8caa-4331-904b-b32e883e1b69 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.726784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c55bd-8caa-4331-904b-b32e883e1b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.727065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c55bd-8caa-4331-904b-b32e883e1b69 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.727755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c55bd-8caa-4331-904b-b32e883e1b69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:14.728066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c55bd-8caa-4331-904b-b32e883e1b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.729469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c55bd-8caa-4331-904b-b32e883e1b69 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.733149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ea81a-caa4-4c91-aeb1-6e1504645423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.733783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ea81a-caa4-4c91-aeb1-6e1504645423 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.734070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ea81a-caa4-4c91-aeb1-6e1504645423 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.734739 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ea81a-caa4-4c91-aeb1-6e1504645423 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.735700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ea81a-caa4-4c91-aeb1-6e1504645423 None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:14.736014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ea81a-caa4-4c91-aeb1-6e1504645423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.737416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ea81a-caa4-4c91-aeb1-6e1504645423 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.746163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 187/365] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:14.747428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c55bd-8caa-4331-904b-b32e883e1b69 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.752673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 179/366] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:14.753019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ea81a-caa4-4c91-aeb1-6e1504645423 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.848887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.886172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ea81a-caa4-4c91-aeb1-6e1504645423 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.889737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c55bd-8caa-4331-904b-b32e883e1b69 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.915051 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.927002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccc09c-fc0d-4967-ad6d-28437397b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.927649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccc09c-fc0d-4967-ad6d-28437397b95d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.928266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccc09c-fc0d-4967-ad6d-28437397b95d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.928551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccc09c-fc0d-4967-ad6d-28437397b95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.932927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33d0f7-5040-4715-ac3f-81844d01c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.933801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33d0f7-5040-4715-ac3f-81844d01c201 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.934254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33d0f7-5040-4715-ac3f-81844d01c201 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.934738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33d0f7-5040-4715-ac3f-81844d01c201 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.948317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33d0f7-5040-4715-ac3f-81844d01c201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-4e33d0f7-5040-4715-ac3f-81844d01c201 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:14.956054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:14.962752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 188/367] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:14.963757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33d0f7-5040-4715-ac3f-81844d01c201 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.018846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccc09c-fc0d-4967-ad6d-28437397b95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.026469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccc09c-fc0d-4967-ad6d-28437397b95d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.099529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33d0f7-5040-4715-ac3f-81844d01c201 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.114915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:15.140145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c59e2-2ae9-4612-8fc5-7c8d39554b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.140848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c59e2-2ae9-4612-8fc5-7c8d39554b6d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.141085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c59e2-2ae9-4612-8fc5-7c8d39554b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.144725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c59e2-2ae9-4612-8fc5-7c8d39554b6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.159688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 180/368] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:15.160550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccc09c-fc0d-4967-ad6d-28437397b95d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.207309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:15.230470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c59e2-2ae9-4612-8fc5-7c8d39554b6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.232100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c59e2-2ae9-4612-8fc5-7c8d39554b6d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.293375 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccc09c-fc0d-4967-ad6d-28437397b95d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.344915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b402196-dde4-487a-b5d4-cca3cf2778c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.345531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b402196-dde4-487a-b5d4-cca3cf2778c0 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.345827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b402196-dde4-487a-b5d4-cca3cf2778c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.346083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b402196-dde4-487a-b5d4-cca3cf2778c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.346793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b402196-dde4-487a-b5d4-cca3cf2778c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:15.347102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b402196-dde4-487a-b5d4-cca3cf2778c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.348495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b402196-dde4-487a-b5d4-cca3cf2778c0 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.357490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 181/369] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:15.358325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b402196-dde4-487a-b5d4-cca3cf2778c0 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.369429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 189/370] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:15.371499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c59e2-2ae9-4612-8fc5-7c8d39554b6d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.399653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:15.462427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:15.477650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b402196-dde4-487a-b5d4-cca3cf2778c0 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.485871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c59e2-2ae9-4612-8fc5-7c8d39554b6d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.517796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6107b8-603a-4f5c-a935-b2fd540ef799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.518445 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6107b8-603a-4f5c-a935-b2fd540ef799 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.518779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6107b8-603a-4f5c-a935-b2fd540ef799 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.519060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6107b8-603a-4f5c-a935-b2fd540ef799 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.520462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6107b8-603a-4f5c-a935-b2fd540ef799 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.521978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6107b8-603a-4f5c-a935-b2fd540ef799 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.524001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 182/371] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:15 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:15.524908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6107b8-603a-4f5c-a935-b2fd540ef799 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.525515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09726b8-44c6-4f4f-8cd2-f8c999aeb353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.526365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09726b8-44c6-4f4f-8cd2-f8c999aeb353 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.526874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09726b8-44c6-4f4f-8cd2-f8c999aeb353 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.527360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09726b8-44c6-4f4f-8cd2-f8c999aeb353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.604008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09726b8-44c6-4f4f-8cd2-f8c999aeb353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.605888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09726b8-44c6-4f4f-8cd2-f8c999aeb353 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.633934 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6107b8-603a-4f5c-a935-b2fd540ef799 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.646932 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:15.683440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cb1cd-9a88-4cbb-be49-30d37c6e7993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.684793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cb1cd-9a88-4cbb-be49-30d37c6e7993 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.684793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cb1cd-9a88-4cbb-be49-30d37c6e7993 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.684793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cb1cd-9a88-4cbb-be49-30d37c6e7993 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.686866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cb1cd-9a88-4cbb-be49-30d37c6e7993 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.689060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cb1cd-9a88-4cbb-be49-30d37c6e7993 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.691109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 183/372] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:15 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:15.691899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cb1cd-9a88-4cbb-be49-30d37c6e7993 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.710015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:15.745606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 190/373] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:30:15.747879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acaab21-21f3-4555-bda3-4cb94e4d24b2 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.748176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acaab21-21f3-4555-bda3-4cb94e4d24b2 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.748447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acaab21-21f3-4555-bda3-4cb94e4d24b2 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.750106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 191/374] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:15.750892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acaab21-21f3-4555-bda3-4cb94e4d24b2 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.846472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cb1cd-9a88-4cbb-be49-30d37c6e7993 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.887894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf33a3f-4258-4282-bda4-2489f0719785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.888463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf33a3f-4258-4282-bda4-2489f0719785 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.888847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf33a3f-4258-4282-bda4-2489f0719785 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.889178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf33a3f-4258-4282-bda4-2489f0719785 None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.891368 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf33a3f-4258-4282-bda4-2489f0719785 None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.892779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf33a3f-4258-4282-bda4-2489f0719785 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.894699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 184/375] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:15 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:15.896830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf33a3f-4258-4282-bda4-2489f0719785 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:15.913869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acaab21-21f3-4555-bda3-4cb94e4d24b2 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:15.951474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:15.959955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c5f6d-be1a-47a7-ab61-d83f77a2876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:15.960900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c5f6d-be1a-47a7-ab61-d83f77a2876c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.961375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c5f6d-be1a-47a7-ab61-d83f77a2876c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.961966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c5f6d-be1a-47a7-ab61-d83f77a2876c None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.963250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c5f6d-be1a-47a7-ab61-d83f77a2876c None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:15.964890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c5f6d-be1a-47a7-ab61-d83f77a2876c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:15.966929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 192/376] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:15 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:15.968264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728a15-cf86-4968-9731-3573b3a0cb8e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:15.969397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728a15-cf86-4968-9731-3573b3a0cb8e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:15.969997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728a15-cf86-4968-9731-3573b3a0cb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:15.992879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-9b728a15-cf86-4968-9731-3573b3a0cb8e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:15.994091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9b728a15-cf86-4968-9731-3573b3a0cb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:16.055472 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:16.064773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf33a3f-4258-4282-bda4-2489f0719785 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.089747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 193/377] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:16.095546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b728a15-cf86-4968-9731-3573b3a0cb8e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.131725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5ef2e-ccd6-4671-b814-d69b90b29dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.132826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5ef2e-ccd6-4671-b814-d69b90b29dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.133373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5ef2e-ccd6-4671-b814-d69b90b29dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.133846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5ef2e-ccd6-4671-b814-d69b90b29dc9 None admin] PATH_INFO: `/v3/projects/0dfb110c668f41649b36f4c725749a3a` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.135261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5ef2e-ccd6-4671-b814-d69b90b29dc9 None admin] RBAC: Authorizing `identity:get_project(project_id=0dfb110c668f41649b36f4c725749a3a)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:16.137026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5ef2e-ccd6-4671-b814-d69b90b29dc9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:16.139517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 185/378] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:15 2022] GET /identity/v3/projects/0dfb110c668f41649b36f4c725749a3a => generated 344 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:16.141005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5ef2e-ccd6-4671-b814-d69b90b29dc9 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.226183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b728a15-cf86-4968-9731-3573b3a0cb8e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.264176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5ef2e-ccd6-4671-b814-d69b90b29dc9 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.268059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6d3ca-57b2-4b77-9d55-844324c7ef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.269711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6d3ca-57b2-4b77-9d55-844324c7ef39 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.270193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6d3ca-57b2-4b77-9d55-844324c7ef39 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.270697 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6d3ca-57b2-4b77-9d55-844324c7ef39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.271656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6d3ca-57b2-4b77-9d55-844324c7ef39 None admin] RBAC: Adding query filter params (group.id=52609d57fd324f898cf5125b6e45e300, role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:16.272163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6d3ca-57b2-4b77-9d55-844324c7ef39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:16.273831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6d3ca-57b2-4b77-9d55-844324c7ef39 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:16.286962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 194/379] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:16 2022] GET /identity/v3/role_assignments?group.id=52609d57fd324f898cf5125b6e45e300&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:16.287973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6d3ca-57b2-4b77-9d55-844324c7ef39 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.288104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:16.314033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359011a2-960d-4297-82f7-6a5bf404ddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.315148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359011a2-960d-4297-82f7-6a5bf404ddfa None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.315699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359011a2-960d-4297-82f7-6a5bf404ddfa None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.316178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359011a2-960d-4297-82f7-6a5bf404ddfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.317245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359011a2-960d-4297-82f7-6a5bf404ddfa None admin] RBAC: Adding query filter params (group.id=52609d57fd324f898cf5125b6e45e300, role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:16.317960 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359011a2-960d-4297-82f7-6a5bf404ddfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:16.320021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359011a2-960d-4297-82f7-6a5bf404ddfa None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:16.326898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 186/380] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:16 2022] GET /identity/v3/role_assignments?group.id=52609d57fd324f898cf5125b6e45e300&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:16.371581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:16.422655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359011a2-960d-4297-82f7-6a5bf404ddfa None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.482349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6d3ca-57b2-4b77-9d55-844324c7ef39 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.535755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663a9d9-0ac0-426b-a647-6980dad1f88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.536417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663a9d9-0ac0-426b-a647-6980dad1f88a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.536700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663a9d9-0ac0-426b-a647-6980dad1f88a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.536990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663a9d9-0ac0-426b-a647-6980dad1f88a None admin] PATH_INFO: `/v3/groups/58eda7b19a4e428e900af77e405f870b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.543820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663a9d9-0ac0-426b-a647-6980dad1f88a None admin] RBAC: Authorizing `identity:get_group(group_id=58eda7b19a4e428e900af77e405f870b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:16.545845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663a9d9-0ac0-426b-a647-6980dad1f88a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:16.549444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 195/381] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:16 2022] GET /identity/v3/groups/58eda7b19a4e428e900af77e405f870b => generated 229 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:16.554993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663a9d9-0ac0-426b-a647-6980dad1f88a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.606052 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359011a2-960d-4297-82f7-6a5bf404ddfa None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.656526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40817a38-c3c3-4530-87c6-99be58455fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.657389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40817a38-c3c3-4530-87c6-99be58455fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.657859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40817a38-c3c3-4530-87c6-99be58455fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.658257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40817a38-c3c3-4530-87c6-99be58455fd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.675235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40817a38-c3c3-4530-87c6-99be58455fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-40817a38-c3c3-4530-87c6-99be58455fd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:16.680930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:16.681858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:16.682783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 187/382] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:16.690214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40817a38-c3c3-4530-87c6-99be58455fd6 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:16.721947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663a9d9-0ac0-426b-a647-6980dad1f88a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:16.730784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:16.761233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:16.766608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0ec60-ea58-49b5-bb7c-02c942064522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:16.767851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0ec60-ea58-49b5-bb7c-02c942064522 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.768347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0ec60-ea58-49b5-bb7c-02c942064522 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.768809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0ec60-ea58-49b5-bb7c-02c942064522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.769712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0ec60-ea58-49b5-bb7c-02c942064522 None admin] RBAC: Adding query filter params (group.id=58eda7b19a4e428e900af77e405f870b, role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=0dfb110c668f41649b36f4c725749a3a) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:16.770183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0ec60-ea58-49b5-bb7c-02c942064522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:16.772938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0ec60-ea58-49b5-bb7c-02c942064522 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:16.779892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 196/383] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:16 2022] GET /identity/v3/role_assignments?group.id=58eda7b19a4e428e900af77e405f870b&scope.project.id=0dfb110c668f41649b36f4c725749a3a&role.id=9e904216086e4a06abf9216b0c863f89 => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:16.815633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b38e39-b189-4328-b134-8366a50aab21 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.816735 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b38e39-b189-4328-b134-8366a50aab21 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.817466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b38e39-b189-4328-b134-8366a50aab21 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.819575 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 197/384] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:16.825324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa98b6-cdc3-46ab-945c-4426163e722d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.825635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa98b6-cdc3-46ab-945c-4426163e722d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.825893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa98b6-cdc3-46ab-945c-4426163e722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.865893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-29aa98b6-cdc3-46ab-945c-4426163e722d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:16.867620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-29aa98b6-cdc3-46ab-945c-4426163e722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:16.917313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40817a38-c3c3-4530-87c6-99be58455fd6 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:17.000271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c6d71-b0ff-4c88-9d5e-05ea8abb752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:17.001131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c6d71-b0ff-4c88-9d5e-05ea8abb752e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.001559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c6d71-b0ff-4c88-9d5e-05ea8abb752e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.002884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c6d71-b0ff-4c88-9d5e-05ea8abb752e None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.003823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c6d71-b0ff-4c88-9d5e-05ea8abb752e None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:17.004560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c6d71-b0ff-4c88-9d5e-05ea8abb752e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:17.007374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c6d71-b0ff-4c88-9d5e-05ea8abb752e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:17.017805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 188/385] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:17.023554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c6d71-b0ff-4c88-9d5e-05ea8abb752e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:17.028941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 198/386] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:17.134579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:17.146477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c6d71-b0ff-4c88-9d5e-05ea8abb752e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:17.195231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc43ff-8753-47a0-b79b-6584c3c13d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:17.195877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc43ff-8753-47a0-b79b-6584c3c13d7f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.196159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc43ff-8753-47a0-b79b-6584c3c13d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.196437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc43ff-8753-47a0-b79b-6584c3c13d7f None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.197617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc43ff-8753-47a0-b79b-6584c3c13d7f None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:17.199103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc43ff-8753-47a0-b79b-6584c3c13d7f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:17.201003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 189/387] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:17 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:17.206668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa98b6-cdc3-46ab-945c-4426163e722d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:17.365104 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:17.371295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa98b6-cdc3-46ab-945c-4426163e722d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:17.425634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f845a-8d39-457d-a45b-7441c850c392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:17.426288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f845a-8d39-457d-a45b-7441c850c392 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.426590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f845a-8d39-457d-a45b-7441c850c392 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.426866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f845a-8d39-457d-a45b-7441c850c392 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.428038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f845a-8d39-457d-a45b-7441c850c392 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:17.429490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f845a-8d39-457d-a45b-7441c850c392 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:17.431540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 199/388] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:17 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:17.439197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc43ff-8753-47a0-b79b-6584c3c13d7f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:17.586540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc43ff-8753-47a0-b79b-6584c3c13d7f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:17.614251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f845a-8d39-457d-a45b-7441c850c392 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:17.644033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9b62f-7540-4427-8704-3dbbbbde90b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:17.644640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9b62f-7540-4427-8704-3dbbbbde90b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.644918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9b62f-7540-4427-8704-3dbbbbde90b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.645194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9b62f-7540-4427-8704-3dbbbbde90b8 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4/groups/52609d57fd324f898cf5125b6e45e300/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.648579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9b62f-7540-4427-8704-3dbbbbde90b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=86e412e829cf47228db2532cb4c7c4e4, group_id=52609d57fd324f898cf5125b6e45e300, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:17.650253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9b62f-7540-4427-8704-3dbbbbde90b8 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:17.752699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 190/389] 10.222.0.82 () {58 vars in 1442 bytes} [Wed Aug 24 03:30:17 2022] PUT /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4/groups/52609d57fd324f898cf5125b6e45e300/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:17.802594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f845a-8d39-457d-a45b-7441c850c392 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:17.850123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61077c-633e-4c88-8ead-bcedb27b81e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:17.850847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61077c-633e-4c88-8ead-bcedb27b81e9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.851171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61077c-633e-4c88-8ead-bcedb27b81e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.851450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61077c-633e-4c88-8ead-bcedb27b81e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.867839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61077c-633e-4c88-8ead-bcedb27b81e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-dd61077c-633e-4c88-8ead-bcedb27b81e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:17.872795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:17.873681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:17.874546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 200/390] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:30:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:17.880369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9b62f-7540-4427-8704-3dbbbbde90b8 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:17.905247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:18.162606 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9b62f-7540-4427-8704-3dbbbbde90b8 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:18.284045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a0a97-f4d3-45c6-acdc-25002a70f280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:18.284596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a0a97-f4d3-45c6-acdc-25002a70f280 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:18.284884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a0a97-f4d3-45c6-acdc-25002a70f280 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:18.285162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a0a97-f4d3-45c6-acdc-25002a70f280 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:18.285920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a0a97-f4d3-45c6-acdc-25002a70f280 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:18.286248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a0a97-f4d3-45c6-acdc-25002a70f280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:18.294532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a0a97-f4d3-45c6-acdc-25002a70f280 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:18.305299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 191/391] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:18.311491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61077c-633e-4c88-8ead-bcedb27b81e9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:18.369486 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:18.519967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61077c-633e-4c88-8ead-bcedb27b81e9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:18.587340 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622323a-55b3-4f7d-9ab5-e6e37d6f3bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:18.588182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622323a-55b3-4f7d-9ab5-e6e37d6f3bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:18.588470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622323a-55b3-4f7d-9ab5-e6e37d6f3bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:18.588749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622323a-55b3-4f7d-9ab5-e6e37d6f3bf0 None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:18.592568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622323a-55b3-4f7d-9ab5-e6e37d6f3bf0 None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:18.592778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622323a-55b3-4f7d-9ab5-e6e37d6f3bf0 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:18.594889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 201/392] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:18 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:18.604493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a0a97-f4d3-45c6-acdc-25002a70f280 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:18.709785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:18.729821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a0a97-f4d3-45c6-acdc-25002a70f280 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:18.774035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748743f8-c02a-4ac8-82f6-ffa44539f711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:18.774641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748743f8-c02a-4ac8-82f6-ffa44539f711 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:18.774922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748743f8-c02a-4ac8-82f6-ffa44539f711 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:18.775194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748743f8-c02a-4ac8-82f6-ffa44539f711 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:18.776340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748743f8-c02a-4ac8-82f6-ffa44539f711 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:18.777779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748743f8-c02a-4ac8-82f6-ffa44539f711 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:18.779929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 192/393] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:18 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:18.794533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622323a-55b3-4f7d-9ab5-e6e37d6f3bf0 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:18.920526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:18.927412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622323a-55b3-4f7d-9ab5-e6e37d6f3bf0 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:18.970148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416756f-d54c-42de-aa9d-1f91ebe1f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:18.971208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416756f-d54c-42de-aa9d-1f91ebe1f13f None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:18.971695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416756f-d54c-42de-aa9d-1f91ebe1f13f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:18.972342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416756f-d54c-42de-aa9d-1f91ebe1f13f None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4/groups/52609d57fd324f898cf5125b6e45e300/roles/e5de7bbccf114283851bcc0028d7cc51` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:18.974604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416756f-d54c-42de-aa9d-1f91ebe1f13f None admin] RBAC: Authorizing `identity:create_grant(project_id=86e412e829cf47228db2532cb4c7c4e4, group_id=52609d57fd324f898cf5125b6e45e300, role_id=e5de7bbccf114283851bcc0028d7cc51)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:18.976057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416756f-d54c-42de-aa9d-1f91ebe1f13f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:19.026785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1106e5e-043e-48ee-a3f2-1b07e3caa6e2 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.027409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1106e5e-043e-48ee-a3f2-1b07e3caa6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.027886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1106e5e-043e-48ee-a3f2-1b07e3caa6e2 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.029686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 193/394] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:19.036248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42268a8-cf57-4831-82a6-852da8cafb2d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.036781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42268a8-cf57-4831-82a6-852da8cafb2d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.037264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42268a8-cf57-4831-82a6-852da8cafb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.057323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-c42268a8-cf57-4831-82a6-852da8cafb2d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:19.058837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c42268a8-cf57-4831-82a6-852da8cafb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:19.104573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 202/395] 10.222.0.82 () {58 vars in 1442 bytes} [Wed Aug 24 03:30:18 2022] PUT /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4/groups/52609d57fd324f898cf5125b6e45e300/roles/e5de7bbccf114283851bcc0028d7cc51 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:19.144785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 194/396] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:30:19.322723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133d9f8-02c3-431f-b8a4-421cb33565c0 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.323372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133d9f8-02c3-431f-b8a4-421cb33565c0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.323876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133d9f8-02c3-431f-b8a4-421cb33565c0 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.325800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 203/397] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:19.331856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286adf3e-24a5-44dd-8b80-18aed499df48 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.332811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286adf3e-24a5-44dd-8b80-18aed499df48 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.333288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286adf3e-24a5-44dd-8b80-18aed499df48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.357629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-286adf3e-24a5-44dd-8b80-18aed499df48 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:19.359167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-286adf3e-24a5-44dd-8b80-18aed499df48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:19.432834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 195/398] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:19.454629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133d9f8-02c3-431f-b8a4-421cb33565c0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:19.569544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133d9f8-02c3-431f-b8a4-421cb33565c0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:19.606458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125db77-c1f9-4050-bdbb-09969e7a1979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:19.607582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125db77-c1f9-4050-bdbb-09969e7a1979 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.608087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125db77-c1f9-4050-bdbb-09969e7a1979 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.608631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125db77-c1f9-4050-bdbb-09969e7a1979 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.621040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125db77-c1f9-4050-bdbb-09969e7a1979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-6125db77-c1f9-4050-bdbb-09969e7a1979 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:19.626978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:19.627917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:19.628730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:19.631368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 204/399] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:30:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:19.637366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286adf3e-24a5-44dd-8b80-18aed499df48 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:19.650882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:19.760585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286adf3e-24a5-44dd-8b80-18aed499df48 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:19.801689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ca259-e20b-4265-9a75-1f1f09b17740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:19.802315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca259-e20b-4265-9a75-1f1f09b17740 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.802614 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca259-e20b-4265-9a75-1f1f09b17740 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.802883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca259-e20b-4265-9a75-1f1f09b17740 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.803561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ca259-e20b-4265-9a75-1f1f09b17740 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:19.803866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ca259-e20b-4265-9a75-1f1f09b17740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:19.805245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ca259-e20b-4265-9a75-1f1f09b17740 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:19.810846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080f344-3ced-4c37-a7d5-4c96353b1469 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.811139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080f344-3ced-4c37-a7d5-4c96353b1469 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.811422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080f344-3ced-4c37-a7d5-4c96353b1469 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.813829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 205/400] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:19.818989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449d4c4-deba-44aa-8561-f028e8ffc841 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.819274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449d4c4-deba-44aa-8561-f028e8ffc841 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.819553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449d4c4-deba-44aa-8561-f028e8ffc841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.819824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 196/401] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:19.825294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ca259-e20b-4265-9a75-1f1f09b17740 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:19.855739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6449d4c4-deba-44aa-8561-f028e8ffc841 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:19.857037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6449d4c4-deba-44aa-8561-f028e8ffc841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:19.871384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:19.920988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:19.940122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 206/402] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:19.944601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449d4c4-deba-44aa-8561-f028e8ffc841 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:19.965476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ca259-e20b-4265-9a75-1f1f09b17740 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.047972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449d4c4-deba-44aa-8561-f028e8ffc841 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.048826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32bf14-6108-46f2-b11d-4d6681e37b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.049583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32bf14-6108-46f2-b11d-4d6681e37b4c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.049875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32bf14-6108-46f2-b11d-4d6681e37b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.050264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32bf14-6108-46f2-b11d-4d6681e37b4c None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.051442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32bf14-6108-46f2-b11d-4d6681e37b4c None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.052900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32bf14-6108-46f2-b11d-4d6681e37b4c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.054493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 197/403] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:19 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:20.064271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32bf14-6108-46f2-b11d-4d6681e37b4c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.090670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2739c-3d62-414f-8a6a-02d461ece9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.092492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2739c-3d62-414f-8a6a-02d461ece9be None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.092777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2739c-3d62-414f-8a6a-02d461ece9be None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.093331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2739c-3d62-414f-8a6a-02d461ece9be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.118429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2739c-3d62-414f-8a6a-02d461ece9be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-bef2739c-3d62-414f-8a6a-02d461ece9be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:20.124155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:20.125194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:20.126037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 207/404] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:20.132781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2739c-3d62-414f-8a6a-02d461ece9be None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.146901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:20.252071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32bf14-6108-46f2-b11d-4d6681e37b4c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.264346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2739c-3d62-414f-8a6a-02d461ece9be None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.297970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917701e-738e-44f3-8de4-f04eb35becfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.298697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917701e-738e-44f3-8de4-f04eb35becfb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.298986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917701e-738e-44f3-8de4-f04eb35becfb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.299437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917701e-738e-44f3-8de4-f04eb35becfb None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.300594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917701e-738e-44f3-8de4-f04eb35becfb None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.302007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834f40e-4333-435f-8746-6b8ab4c37854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.302140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917701e-738e-44f3-8de4-f04eb35becfb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.302852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834f40e-4333-435f-8746-6b8ab4c37854 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.303231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834f40e-4333-435f-8746-6b8ab4c37854 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.303599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834f40e-4333-435f-8746-6b8ab4c37854 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.304050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 198/405] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:20 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:20.304597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834f40e-4333-435f-8746-6b8ab4c37854 None admin] RBAC: Adding query filter params (name=member) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:20.304977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917701e-738e-44f3-8de4-f04eb35becfb None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.305276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834f40e-4333-435f-8746-6b8ab4c37854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.306910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834f40e-4333-435f-8746-6b8ab4c37854 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.316694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 208/406] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:20.317742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834f40e-4333-435f-8746-6b8ab4c37854 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.317911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:20.413760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:20.425699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917701e-738e-44f3-8de4-f04eb35becfb None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.455046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834f40e-4333-435f-8746-6b8ab4c37854 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.463576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584975b-0c83-418b-a279-4b54e4ed5ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.464520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584975b-0c83-418b-a279-4b54e4ed5ecf None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.464996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584975b-0c83-418b-a279-4b54e4ed5ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.465465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584975b-0c83-418b-a279-4b54e4ed5ecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.477217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584975b-0c83-418b-a279-4b54e4ed5ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-d584975b-0c83-418b-a279-4b54e4ed5ecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:20.482921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:20.483857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:30:20.485063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:20.487855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 199/407] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:30:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:20.488926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584975b-0c83-418b-a279-4b54e4ed5ecf None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.493973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693494d-cdde-414c-a610-c3d45a117ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.493973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693494d-cdde-414c-a610-c3d45a117ffa None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.494196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693494d-cdde-414c-a610-c3d45a117ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.494404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693494d-cdde-414c-a610-c3d45a117ffa None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/groups/52609d57fd324f898cf5125b6e45e300/roles/e5de7bbccf114283851bcc0028d7cc51` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.496720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693494d-cdde-414c-a610-c3d45a117ffa None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b2151f913144fd856a4dd3b1181fe4, group_id=52609d57fd324f898cf5125b6e45e300, role_id=e5de7bbccf114283851bcc0028d7cc51)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.498187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693494d-cdde-414c-a610-c3d45a117ffa None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.581271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:20.624449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 209/408] 10.222.0.82 () {58 vars in 1442 bytes} [Wed Aug 24 03:30:20 2022] PUT /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/groups/52609d57fd324f898cf5125b6e45e300/roles/e5de7bbccf114283851bcc0028d7cc51 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:20.625926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693494d-cdde-414c-a610-c3d45a117ffa None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.628985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584975b-0c83-418b-a279-4b54e4ed5ecf None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.684071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3987b-02ff-414d-bd6c-4c75967d675b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.684692 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3987b-02ff-414d-bd6c-4c75967d675b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.685040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3987b-02ff-414d-bd6c-4c75967d675b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.685314 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3987b-02ff-414d-bd6c-4c75967d675b None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.687030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3987b-02ff-414d-bd6c-4c75967d675b None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.689201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3987b-02ff-414d-bd6c-4c75967d675b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.691748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 200/409] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:20 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:20.700028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3987b-02ff-414d-bd6c-4c75967d675b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.823794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693494d-cdde-414c-a610-c3d45a117ffa None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.831071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3987b-02ff-414d-bd6c-4c75967d675b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.856062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:20.865434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3ee33-837b-433f-8f61-9bccbdac82f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.866089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3ee33-837b-433f-8f61-9bccbdac82f6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.866375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3ee33-837b-433f-8f61-9bccbdac82f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.866681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3ee33-837b-433f-8f61-9bccbdac82f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.871167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3ee33-837b-433f-8f61-9bccbdac82f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:20.871167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3ee33-837b-433f-8f61-9bccbdac82f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.871167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3ee33-837b-433f-8f61-9bccbdac82f6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.871379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7d4ef-7669-4dc7-a853-7b1d8f5df2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.871465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7d4ef-7669-4dc7-a853-7b1d8f5df2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.871730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7d4ef-7669-4dc7-a853-7b1d8f5df2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.872005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7d4ef-7669-4dc7-a853-7b1d8f5df2d5 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.873141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7d4ef-7669-4dc7-a853-7b1d8f5df2d5 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.874637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7d4ef-7669-4dc7-a853-7b1d8f5df2d5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.876831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 201/410] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:20 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:20.877985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122abc4-1b30-49d1-b944-272aaecc40d3 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.878272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122abc4-1b30-49d1-b944-272aaecc40d3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.879482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122abc4-1b30-49d1-b944-272aaecc40d3 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.879908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 202/411] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:20.882473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 210/412] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:20.884880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3ee33-837b-433f-8f61-9bccbdac82f6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.885264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35bbce-bfc3-4aa9-bf50-3190e9c7211c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.885550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35bbce-bfc3-4aa9-bf50-3190e9c7211c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.885858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35bbce-bfc3-4aa9-bf50-3190e9c7211c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.894225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:20.908564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-3d35bbce-bfc3-4aa9-bf50-3190e9c7211c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:20.909938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3d35bbce-bfc3-4aa9-bf50-3190e9c7211c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:21.004102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 203/413] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:30:21.004723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35bbce-bfc3-4aa9-bf50-3190e9c7211c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.037187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3ee33-837b-433f-8f61-9bccbdac82f6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.077961 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e9b8c-a0b5-4482-8e52-b4744941c160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.079480 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9b8c-a0b5-4482-8e52-b4744941c160 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.079480 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9b8c-a0b5-4482-8e52-b4744941c160 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.079762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9b8c-a0b5-4482-8e52-b4744941c160 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/groups/52609d57fd324f898cf5125b6e45e300/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.081981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e9b8c-a0b5-4482-8e52-b4744941c160 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8b2151f913144fd856a4dd3b1181fe4, group_id=52609d57fd324f898cf5125b6e45e300, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.083762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e9b8c-a0b5-4482-8e52-b4744941c160 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.168773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35bbce-bfc3-4aa9-bf50-3190e9c7211c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.208276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 211/414] 10.222.0.82 () {58 vars in 1442 bytes} [Wed Aug 24 03:30:20 2022] PUT /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4/groups/52609d57fd324f898cf5125b6e45e300/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:21.217087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185eda0f-309d-49a6-a6ee-f08eb8560674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.217764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185eda0f-309d-49a6-a6ee-f08eb8560674 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.218047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185eda0f-309d-49a6-a6ee-f08eb8560674 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.218322 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185eda0f-309d-49a6-a6ee-f08eb8560674 None admin] PATH_INFO: `/v3/groups/58eda7b19a4e428e900af77e405f870b` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.221063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185eda0f-309d-49a6-a6ee-f08eb8560674 None admin] RBAC: Authorizing `identity:get_group(group_id=58eda7b19a4e428e900af77e405f870b)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.222666 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185eda0f-309d-49a6-a6ee-f08eb8560674 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.224547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 204/415] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:21 2022] GET /identity/v3/groups/58eda7b19a4e428e900af77e405f870b => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:21.232615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e9b8c-a0b5-4482-8e52-b4744941c160 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.308763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:21.327480 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185eda0f-309d-49a6-a6ee-f08eb8560674 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.331678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:21.391343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e9b8c-a0b5-4482-8e52-b4744941c160 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.439121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d00fc-d961-417e-8349-7e76fb94e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.440713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d00fc-d961-417e-8349-7e76fb94e375 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.441196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d00fc-d961-417e-8349-7e76fb94e375 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.441965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d00fc-d961-417e-8349-7e76fb94e375 None admin] PATH_INFO: `/v3/projects/0dfb110c668f41649b36f4c725749a3a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.444096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d00fc-d961-417e-8349-7e76fb94e375 None admin] RBAC: Authorizing `identity:get_project(project_id=0dfb110c668f41649b36f4c725749a3a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.446148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d00fc-d961-417e-8349-7e76fb94e375 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.449720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 212/416] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:21 2022] GET /identity/v3/projects/0dfb110c668f41649b36f4c725749a3a => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:21.455815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d00fc-d961-417e-8349-7e76fb94e375 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.457069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185eda0f-309d-49a6-a6ee-f08eb8560674 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.495869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e896149-119c-4a5f-810d-798e5441bf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.496508 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e896149-119c-4a5f-810d-798e5441bf29 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.496798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e896149-119c-4a5f-810d-798e5441bf29 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.497061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e896149-119c-4a5f-810d-798e5441bf29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.524646 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e896149-119c-4a5f-810d-798e5441bf29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-2e896149-119c-4a5f-810d-798e5441bf29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:21.530270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:21.531305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:21.532245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 205/417] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:21.540026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e896149-119c-4a5f-810d-798e5441bf29 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.613216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:21.668416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d00fc-d961-417e-8349-7e76fb94e375 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.674159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e896149-119c-4a5f-810d-798e5441bf29 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.701523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:21.721516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94191d69-68f9-4db3-9106-afd22cf25375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.722386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191d69-68f9-4db3-9106-afd22cf25375 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.722657 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191d69-68f9-4db3-9106-afd22cf25375 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.722864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191d69-68f9-4db3-9106-afd22cf25375 None admin] PATH_INFO: `/v3/projects/0dfb110c668f41649b36f4c725749a3a/groups/58eda7b19a4e428e900af77e405f870b/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.725495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94191d69-68f9-4db3-9106-afd22cf25375 None admin] RBAC: Authorizing `identity:create_grant(project_id=0dfb110c668f41649b36f4c725749a3a, group_id=58eda7b19a4e428e900af77e405f870b, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.728878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94191d69-68f9-4db3-9106-afd22cf25375 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.729327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63171def-e604-49f7-b0a2-f5ba291a5cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.730233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63171def-e604-49f7-b0a2-f5ba291a5cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.731063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63171def-e604-49f7-b0a2-f5ba291a5cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.731755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63171def-e604-49f7-b0a2-f5ba291a5cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.733181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63171def-e604-49f7-b0a2-f5ba291a5cd5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:21.733984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63171def-e604-49f7-b0a2-f5ba291a5cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:21.737948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63171def-e604-49f7-b0a2-f5ba291a5cd5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:21.757360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 206/418] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:21.757360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391018aa-a041-4081-be43-667bc5e656b4 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.757360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391018aa-a041-4081-be43-667bc5e656b4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.757360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391018aa-a041-4081-be43-667bc5e656b4 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.758730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 207/419] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:21.759938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391018aa-a041-4081-be43-667bc5e656b4 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:21.816220 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 213/420] 10.222.0.82 () {58 vars in 1442 bytes} [Wed Aug 24 03:30:21 2022] PUT /identity/v3/projects/0dfb110c668f41649b36f4c725749a3a/groups/58eda7b19a4e428e900af77e405f870b/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:21.818047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dde08-12aa-4f4d-af3f-966ac5e4cea9 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.818709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dde08-12aa-4f4d-af3f-966ac5e4cea9 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.818709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dde08-12aa-4f4d-af3f-966ac5e4cea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:21.848497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-084dde08-12aa-4f4d-af3f-966ac5e4cea9 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:21.851285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-084dde08-12aa-4f4d-af3f-966ac5e4cea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:21.909443 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391018aa-a041-4081-be43-667bc5e656b4 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:21.940302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:21.955107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e4867-2a2f-4326-a1b8-547d0dfb4c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:21.955965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e4867-2a2f-4326-a1b8-547d0dfb4c00 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:21.956387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e4867-2a2f-4326-a1b8-547d0dfb4c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:21.957273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e4867-2a2f-4326-a1b8-547d0dfb4c00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.015264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 214/421] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:22.069169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e4867-2a2f-4326-a1b8-547d0dfb4c00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:22.070938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e4867-2a2f-4326-a1b8-547d0dfb4c00 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:22.193459 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 208/422] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:22.195275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dde08-12aa-4f4d-af3f-966ac5e4cea9 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:22.334007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dde08-12aa-4f4d-af3f-966ac5e4cea9 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:22.344233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:22.374056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471899a1-0940-4786-833a-715d29e83fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:22.374779 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471899a1-0940-4786-833a-715d29e83fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.375117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471899a1-0940-4786-833a-715d29e83fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.375392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471899a1-0940-4786-833a-715d29e83fa3 None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.379192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471899a1-0940-4786-833a-715d29e83fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:22.380799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471899a1-0940-4786-833a-715d29e83fa3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:22.382822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 215/423] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:22 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:22.382936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e4867-2a2f-4326-a1b8-547d0dfb4c00 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:22.389856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471899a1-0940-4786-833a-715d29e83fa3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:22.444845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:22.566426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471899a1-0940-4786-833a-715d29e83fa3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:22.602040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e4867-2a2f-4326-a1b8-547d0dfb4c00 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:22.643361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564dbe7-615e-4355-ac67-6a104e3bb92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:22.644005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564dbe7-615e-4355-ac67-6a104e3bb92f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.644290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564dbe7-615e-4355-ac67-6a104e3bb92f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.644569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564dbe7-615e-4355-ac67-6a104e3bb92f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.647233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644695a5-4a69-4d20-b282-fae9eafcb45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:22.647874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644695a5-4a69-4d20-b282-fae9eafcb45d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.648158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644695a5-4a69-4d20-b282-fae9eafcb45d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.648436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644695a5-4a69-4d20-b282-fae9eafcb45d None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.650578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644695a5-4a69-4d20-b282-fae9eafcb45d None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:22.654275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644695a5-4a69-4d20-b282-fae9eafcb45d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:22.656282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 216/424] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:22 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:22.663558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644695a5-4a69-4d20-b282-fae9eafcb45d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:22.665163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564dbe7-615e-4355-ac67-6a104e3bb92f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-a564dbe7-615e-4355-ac67-6a104e3bb92f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:22.673401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:22.674350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:22.675187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:22.680216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 209/425] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:30:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:22.685416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564dbe7-615e-4355-ac67-6a104e3bb92f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:22.752719 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:22.840271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:22.858839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644695a5-4a69-4d20-b282-fae9eafcb45d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:22.878204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564dbe7-615e-4355-ac67-6a104e3bb92f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:22.931592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5c440-2786-41dc-9b66-3e955102d23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:22.932272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5c440-2786-41dc-9b66-3e955102d23f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.932562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5c440-2786-41dc-9b66-3e955102d23f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.932838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5c440-2786-41dc-9b66-3e955102d23f None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.933562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5c440-2786-41dc-9b66-3e955102d23f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:22.933869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5c440-2786-41dc-9b66-3e955102d23f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:22.935291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5c440-2786-41dc-9b66-3e955102d23f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:22.936635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41c8fb-cbb1-4208-abae-f65e6d8b76d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:22.937259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41c8fb-cbb1-4208-abae-f65e6d8b76d1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.937576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41c8fb-cbb1-4208-abae-f65e6d8b76d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.938523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41c8fb-cbb1-4208-abae-f65e6d8b76d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.939325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41c8fb-cbb1-4208-abae-f65e6d8b76d1 None admin] RBAC: Adding query filter params (group.id=52609d57fd324f898cf5125b6e45e300, role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=86e412e829cf47228db2532cb4c7c4e4) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:22.939706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41c8fb-cbb1-4208-abae-f65e6d8b76d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:22.941165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41c8fb-cbb1-4208-abae-f65e6d8b76d1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:22.945508 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 210/426] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:22.951493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 217/427] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:22 2022] GET /identity/v3/role_assignments?group.id=52609d57fd324f898cf5125b6e45e300&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 611 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:22.956523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5c440-2786-41dc-9b66-3e955102d23f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:23.103654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5c440-2786-41dc-9b66-3e955102d23f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:23.139069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6e64a-346d-43eb-bf01-55fcb180d5c7 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.139464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6e64a-346d-43eb-bf01-55fcb180d5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.139869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6e64a-346d-43eb-bf01-55fcb180d5c7 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.141606 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529fe92-3da8-4f41-8780-76dbe1682ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:23.142175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 218/428] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:23.142293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529fe92-3da8-4f41-8780-76dbe1682ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.142583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529fe92-3da8-4f41-8780-76dbe1682ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.142859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529fe92-3da8-4f41-8780-76dbe1682ba9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.149039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638e6c0-fec4-4b16-8115-58c2f00c1e89 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.149333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638e6c0-fec4-4b16-8115-58c2f00c1e89 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.149619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638e6c0-fec4-4b16-8115-58c2f00c1e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.165659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:23.169651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d638e6c0-fec4-4b16-8115-58c2f00c1e89 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:23.171421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d638e6c0-fec4-4b16-8115-58c2f00c1e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:23.250562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529fe92-3da8-4f41-8780-76dbe1682ba9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:23.252095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529fe92-3da8-4f41-8780-76dbe1682ba9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:23.276512 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 219/429] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:23.372255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 211/430] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:23.378564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638e6c0-fec4-4b16-8115-58c2f00c1e89 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:23.439635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:23.533525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638e6c0-fec4-4b16-8115-58c2f00c1e89 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:23.571972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529fe92-3da8-4f41-8780-76dbe1682ba9 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:23.593878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ec0fd-1d36-4c0d-bf49-70541bbf1e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:23.595181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ec0fd-1d36-4c0d-bf49-70541bbf1e5b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.595830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ec0fd-1d36-4c0d-bf49-70541bbf1e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.596307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ec0fd-1d36-4c0d-bf49-70541bbf1e5b None admin] PATH_INFO: `/v3/projects/86e412e829cf47228db2532cb4c7c4e4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.598255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ec0fd-1d36-4c0d-bf49-70541bbf1e5b None admin] RBAC: Authorizing `identity:get_project(project_id=86e412e829cf47228db2532cb4c7c4e4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:23.601078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ec0fd-1d36-4c0d-bf49-70541bbf1e5b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:23.604349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 220/431] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:23 2022] GET /identity/v3/projects/86e412e829cf47228db2532cb4c7c4e4 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:23.614352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ec0fd-1d36-4c0d-bf49-70541bbf1e5b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:23.628389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:23.704788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529fe92-3da8-4f41-8780-76dbe1682ba9 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:23.765418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bee50-3c05-43ab-9e69-7ba148eec5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:23.765712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ec0fd-1d36-4c0d-bf49-70541bbf1e5b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:23.766190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bee50-3c05-43ab-9e69-7ba148eec5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.766778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bee50-3c05-43ab-9e69-7ba148eec5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.767065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bee50-3c05-43ab-9e69-7ba148eec5a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.779256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:23.784033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bee50-3c05-43ab-9e69-7ba148eec5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-1c7bee50-3c05-43ab-9e69-7ba148eec5a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:23.788982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:23.789887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:23.790792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 212/432] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:30:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:23.796362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265702eb-76fb-44ea-bc38-914270353b28 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.796362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265702eb-76fb-44ea-bc38-914270353b28 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.796362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265702eb-76fb-44ea-bc38-914270353b28 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.796362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 213/433] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:23.797613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265702eb-76fb-44ea-bc38-914270353b28 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:23.826004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052d261-b3e3-401e-b010-bbe156eebfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:23.826004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d261-b3e3-401e-b010-bbe156eebfbe None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.826004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d261-b3e3-401e-b010-bbe156eebfbe None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.826004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d261-b3e3-401e-b010-bbe156eebfbe None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.826407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052d261-b3e3-401e-b010-bbe156eebfbe None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:23.827061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052d261-b3e3-401e-b010-bbe156eebfbe None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:23.829438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 221/434] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:23 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:23.834312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd61a8c-8037-4549-9d01-6d323053498d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.834312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd61a8c-8037-4549-9d01-6d323053498d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.834312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd61a8c-8037-4549-9d01-6d323053498d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.853193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-abd61a8c-8037-4549-9d01-6d323053498d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:23.854841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-abd61a8c-8037-4549-9d01-6d323053498d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:23.944966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265702eb-76fb-44ea-bc38-914270353b28 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:23.945262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 222/435] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:23.946290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd61a8c-8037-4549-9d01-6d323053498d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:23.954101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:23.992971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8d753-11e4-4d60-864c-ef3e20a4fbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:23.993925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8d753-11e4-4d60-864c-ef3e20a4fbbd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.994390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8d753-11e4-4d60-864c-ef3e20a4fbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.994857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8d753-11e4-4d60-864c-ef3e20a4fbbd None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.995760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8d753-11e4-4d60-864c-ef3e20a4fbbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:23.996293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8d753-11e4-4d60-864c-ef3e20a4fbbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:23.998645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8d753-11e4-4d60-864c-ef3e20a4fbbd None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:24.011091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 214/436] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:24.027594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8d753-11e4-4d60-864c-ef3e20a4fbbd None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:24.097410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd61a8c-8037-4549-9d01-6d323053498d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:24.104167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:24.137099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3679-d434-407e-8e62-a2faef0402b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:24.137711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3679-d434-407e-8e62-a2faef0402b6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.137990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3679-d434-407e-8e62-a2faef0402b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.138260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3679-d434-407e-8e62-a2faef0402b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.139111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d3679-d434-407e-8e62-a2faef0402b6 None admin] RBAC: Adding query filter params (group.id=52609d57fd324f898cf5125b6e45e300, role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=86e412e829cf47228db2532cb4c7c4e4) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:24.139559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d3679-d434-407e-8e62-a2faef0402b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:24.141679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d3679-d434-407e-8e62-a2faef0402b6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:24.148928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 223/437] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:23 2022] GET /identity/v3/role_assignments?group.id=52609d57fd324f898cf5125b6e45e300&scope.project.id=86e412e829cf47228db2532cb4c7c4e4&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:24.221952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8d753-11e4-4d60-864c-ef3e20a4fbbd None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:24.275523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f730c1-171a-4ca8-baff-9f6a9df87ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:24.276598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f730c1-171a-4ca8-baff-9f6a9df87ebc None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.277088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f730c1-171a-4ca8-baff-9f6a9df87ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.277571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f730c1-171a-4ca8-baff-9f6a9df87ebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.299994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3679-d434-407e-8e62-a2faef0402b6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:24.325086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:24.370897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f730c1-171a-4ca8-baff-9f6a9df87ebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:24.372892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f730c1-171a-4ca8-baff-9f6a9df87ebc None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:24.479484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3679-d434-407e-8e62-a2faef0402b6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:24.518655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 215/438] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:24.518655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440f66b-b64d-4fc7-97a5-f5807d0922db None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.518655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440f66b-b64d-4fc7-97a5-f5807d0922db None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.518655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440f66b-b64d-4fc7-97a5-f5807d0922db None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.519460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 216/439] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:24.521754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440f66b-b64d-4fc7-97a5-f5807d0922db None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:24.525559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3118b-a508-4a6b-bb29-1abe9f85fe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:24.526586 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3118b-a508-4a6b-bb29-1abe9f85fe77 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.526951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3118b-a508-4a6b-bb29-1abe9f85fe77 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.527229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3118b-a508-4a6b-bb29-1abe9f85fe77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.548228 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3118b-a508-4a6b-bb29-1abe9f85fe77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-bea3118b-a508-4a6b-bb29-1abe9f85fe77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:24.554746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:24.555688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:30:24.556497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:24.557123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 224/440] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:24.558115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458ec8f-8db6-4d7a-a4c6-f0ce2bd83c5a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.558456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458ec8f-8db6-4d7a-a4c6-f0ce2bd83c5a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.559011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458ec8f-8db6-4d7a-a4c6-f0ce2bd83c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.585588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e458ec8f-8db6-4d7a-a4c6-f0ce2bd83c5a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:24.587908 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e458ec8f-8db6-4d7a-a4c6-f0ce2bd83c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:24.623800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:24.625845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:24.658524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440f66b-b64d-4fc7-97a5-f5807d0922db None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:24.674424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 225/441] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:24.675310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458ec8f-8db6-4d7a-a4c6-f0ce2bd83c5a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:24.697881 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c46d1f-64cf-4a89-854c-bcb5b19ecff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:24.699323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c46d1f-64cf-4a89-854c-bcb5b19ecff3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.699817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c46d1f-64cf-4a89-854c-bcb5b19ecff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.700533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c46d1f-64cf-4a89-854c-bcb5b19ecff3 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.702340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c46d1f-64cf-4a89-854c-bcb5b19ecff3 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:24.704497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c46d1f-64cf-4a89-854c-bcb5b19ecff3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:24.706153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 217/442] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:24 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:24.711568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c46d1f-64cf-4a89-854c-bcb5b19ecff3 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:24.804642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458ec8f-8db6-4d7a-a4c6-f0ce2bd83c5a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:24.838077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c46d1f-64cf-4a89-854c-bcb5b19ecff3 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:24.844452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e919b9-6d86-4b7f-8afd-cdc91c2dfdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:24.845123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e919b9-6d86-4b7f-8afd-cdc91c2dfdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.845407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e919b9-6d86-4b7f-8afd-cdc91c2dfdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.845686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e919b9-6d86-4b7f-8afd-cdc91c2dfdd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.846365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e919b9-6d86-4b7f-8afd-cdc91c2dfdd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:24.846714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e919b9-6d86-4b7f-8afd-cdc91c2dfdd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:24.848105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e919b9-6d86-4b7f-8afd-cdc91c2dfdd1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:24.857442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 226/443] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:30:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:24.864493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e919b9-6d86-4b7f-8afd-cdc91c2dfdd1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:24.879616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c751584-6e7f-4970-8102-93b0e3fa4203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:24.880187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c751584-6e7f-4970-8102-93b0e3fa4203 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:24.880483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c751584-6e7f-4970-8102-93b0e3fa4203 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:24.880747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c751584-6e7f-4970-8102-93b0e3fa4203 None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:24.882164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c751584-6e7f-4970-8102-93b0e3fa4203 None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:24.883662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c751584-6e7f-4970-8102-93b0e3fa4203 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:24.885938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 218/444] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:24 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:24.890603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:24.892326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c751584-6e7f-4970-8102-93b0e3fa4203 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:24.896905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:24.987378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e919b9-6d86-4b7f-8afd-cdc91c2dfdd1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.007767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c751584-6e7f-4970-8102-93b0e3fa4203 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.031582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35053-c589-4d41-8bf8-6d55c3506a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.032207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35053-c589-4d41-8bf8-6d55c3506a82 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.032488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35053-c589-4d41-8bf8-6d55c3506a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.032761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35053-c589-4d41-8bf8-6d55c3506a82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.042249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec1918-2495-4697-b33b-f2e4dbd8ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.042889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec1918-2495-4697-b33b-f2e4dbd8ae42 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.043172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec1918-2495-4697-b33b-f2e4dbd8ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.043447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec1918-2495-4697-b33b-f2e4dbd8ae42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.045118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec1918-2495-4697-b33b-f2e4dbd8ae42 None admin] RBAC: Adding query filter params (group.id=52609d57fd324f898cf5125b6e45e300, role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:25.045118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec1918-2495-4697-b33b-f2e4dbd8ae42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.045895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec1918-2495-4697-b33b-f2e4dbd8ae42 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.056706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 219/445] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:24 2022] GET /identity/v3/role_assignments?group.id=52609d57fd324f898cf5125b6e45e300&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:25.056706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec1918-2495-4697-b33b-f2e4dbd8ae42 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.138281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db35053-c589-4d41-8bf8-6d55c3506a82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.141045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db35053-c589-4d41-8bf8-6d55c3506a82 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.158211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:25.170945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:25.197223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec1918-2495-4697-b33b-f2e4dbd8ae42 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.245742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432680e5-7cb5-45b2-a0d3-d019696947bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.246528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432680e5-7cb5-45b2-a0d3-d019696947bb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.246822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432680e5-7cb5-45b2-a0d3-d019696947bb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.247102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432680e5-7cb5-45b2-a0d3-d019696947bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.262607 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432680e5-7cb5-45b2-a0d3-d019696947bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-432680e5-7cb5-45b2-a0d3-d019696947bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:25.269567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:25.270569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:30:25.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:25.273596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 220/446] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:25.279084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 227/447] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:30:25.280126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432680e5-7cb5-45b2-a0d3-d019696947bb None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.283979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35053-c589-4d41-8bf8-6d55c3506a82 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.391717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35053-c589-4d41-8bf8-6d55c3506a82 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.394257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432680e5-7cb5-45b2-a0d3-d019696947bb None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.417336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:25.423678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:25.431434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107d05d-a199-49da-92b0-25ab936251d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.432174 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107d05d-a199-49da-92b0-25ab936251d3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.432273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe3f3c-c373-4e80-bcc0-beacadd8fb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.432485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107d05d-a199-49da-92b0-25ab936251d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.432703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe3f3c-c373-4e80-bcc0-beacadd8fb18 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.432803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107d05d-a199-49da-92b0-25ab936251d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.432989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe3f3c-c373-4e80-bcc0-beacadd8fb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.433268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe3f3c-c373-4e80-bcc0-beacadd8fb18 None admin] PATH_INFO: `/v3/projects/f8b2151f913144fd856a4dd3b1181fe4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.433362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107d05d-a199-49da-92b0-25ab936251d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:25.433663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107d05d-a199-49da-92b0-25ab936251d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.434409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe3f3c-c373-4e80-bcc0-beacadd8fb18 None admin] RBAC: Authorizing `identity:get_project(project_id=f8b2151f913144fd856a4dd3b1181fe4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.435089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107d05d-a199-49da-92b0-25ab936251d3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.436086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe3f3c-c373-4e80-bcc0-beacadd8fb18 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.437945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 228/448] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/projects/f8b2151f913144fd856a4dd3b1181fe4 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:25.443576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe3f3c-c373-4e80-bcc0-beacadd8fb18 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.443724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 221/449] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:25.449212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107d05d-a199-49da-92b0-25ab936251d3 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.551795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe3f3c-c373-4e80-bcc0-beacadd8fb18 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.562053 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107d05d-a199-49da-92b0-25ab936251d3 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.589340 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e656a2-15c3-447b-8f18-5fa6f203c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.590091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e656a2-15c3-447b-8f18-5fa6f203c707 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.590376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e656a2-15c3-447b-8f18-5fa6f203c707 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.590682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e656a2-15c3-447b-8f18-5fa6f203c707 None admin] PATH_INFO: `/v3/groups/52609d57fd324f898cf5125b6e45e300` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.591834 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e656a2-15c3-447b-8f18-5fa6f203c707 None admin] RBAC: Authorizing `identity:get_group(group_id=52609d57fd324f898cf5125b6e45e300)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.593290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e656a2-15c3-447b-8f18-5fa6f203c707 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.596234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 229/450] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/groups/52609d57fd324f898cf5125b6e45e300 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:25.600212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b0478-5a49-46ed-ab6e-068abbceafe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.600847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b0478-5a49-46ed-ab6e-068abbceafe9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.601116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b0478-5a49-46ed-ab6e-068abbceafe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.601397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b0478-5a49-46ed-ab6e-068abbceafe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.603537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e656a2-15c3-447b-8f18-5fa6f203c707 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.663938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:25.668650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:25.675870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b0478-5a49-46ed-ab6e-068abbceafe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.677358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b0478-5a49-46ed-ab6e-068abbceafe9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.715660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e656a2-15c3-447b-8f18-5fa6f203c707 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.751400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8430f6-0be9-41e7-82e4-9f7c003c044d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.752014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8430f6-0be9-41e7-82e4-9f7c003c044d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.752300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8430f6-0be9-41e7-82e4-9f7c003c044d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.752578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8430f6-0be9-41e7-82e4-9f7c003c044d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.753320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8430f6-0be9-41e7-82e4-9f7c003c044d None admin] RBAC: Adding query filter params (group.id=52609d57fd324f898cf5125b6e45e300, role.id=5bf6686ae6624e84aba335589d8a98a4, scope.project.id=f8b2151f913144fd856a4dd3b1181fe4) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:25.753638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8430f6-0be9-41e7-82e4-9f7c003c044d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.755179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8430f6-0be9-41e7-82e4-9f7c003c044d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.761779 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 230/451] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/role_assignments?group.id=52609d57fd324f898cf5125b6e45e300&scope.project.id=f8b2151f913144fd856a4dd3b1181fe4&role.id=5bf6686ae6624e84aba335589d8a98a4 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:25.786608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 222/452] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:30:25.792829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8430f6-0be9-41e7-82e4-9f7c003c044d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.913487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8430f6-0be9-41e7-82e4-9f7c003c044d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:25.935136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:25.952218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d8070-e856-405a-9789-3df4ade08d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:25.954388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d8070-e856-405a-9789-3df4ade08d09 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.954388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d8070-e856-405a-9789-3df4ade08d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.954388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d8070-e856-405a-9789-3df4ade08d09 None admin] PATH_INFO: `/v3/projects/0dfb110c668f41649b36f4c725749a3a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.955473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d8070-e856-405a-9789-3df4ade08d09 None admin] RBAC: Authorizing `identity:get_project(project_id=0dfb110c668f41649b36f4c725749a3a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:25.956942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d8070-e856-405a-9789-3df4ade08d09 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:25.959269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 231/453] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/projects/0dfb110c668f41649b36f4c725749a3a => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:25.964176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b0478-5a49-46ed-ab6e-068abbceafe9 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:26.072358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:26.082935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b0478-5a49-46ed-ab6e-068abbceafe9 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:26.121019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb91f76-51cf-45cb-910f-8d7f70801121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:26.121672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb91f76-51cf-45cb-910f-8d7f70801121 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:26.121959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb91f76-51cf-45cb-910f-8d7f70801121 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:26.122237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb91f76-51cf-45cb-910f-8d7f70801121 None admin] PATH_INFO: `/v3/groups/58eda7b19a4e428e900af77e405f870b` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:26.123415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb91f76-51cf-45cb-910f-8d7f70801121 None admin] RBAC: Authorizing `identity:get_group(group_id=58eda7b19a4e428e900af77e405f870b)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:26.124871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb91f76-51cf-45cb-910f-8d7f70801121 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:26.127051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 223/454] 10.222.0.82 () {56 vars in 1180 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/groups/58eda7b19a4e428e900af77e405f870b => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:26.132990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d8070-e856-405a-9789-3df4ade08d09 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:26.243733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d8070-e856-405a-9789-3df4ade08d09 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:26.279797 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933985f-96ba-4ae2-9185-91a0fb31e027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:26.280433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933985f-96ba-4ae2-9185-91a0fb31e027 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:26.280728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933985f-96ba-4ae2-9185-91a0fb31e027 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:26.281003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933985f-96ba-4ae2-9185-91a0fb31e027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:26.282966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933985f-96ba-4ae2-9185-91a0fb31e027 None admin] RBAC: Adding query filter params (group.id=58eda7b19a4e428e900af77e405f870b, role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=0dfb110c668f41649b36f4c725749a3a) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:26.283284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933985f-96ba-4ae2-9185-91a0fb31e027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:26.284749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933985f-96ba-4ae2-9185-91a0fb31e027 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:26.290838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 232/455] 10.222.0.82 () {56 vars in 1376 bytes} [Wed Aug 24 03:30:26 2022] GET /identity/v3/role_assignments?group.id=58eda7b19a4e428e900af77e405f870b&scope.project.id=0dfb110c668f41649b36f4c725749a3a&role.id=9e904216086e4a06abf9216b0c863f89 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:28.642358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4033-a86a-45cc-8ca2-555611162bf8 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.642358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4033-a86a-45cc-8ca2-555611162bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.642358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4033-a86a-45cc-8ca2-555611162bf8 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.643709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 224/456] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:28.648413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a902dc-eef1-4b19-a95f-c33d74be1002 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.648957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a902dc-eef1-4b19-a95f-c33d74be1002 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.648957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a902dc-eef1-4b19-a95f-c33d74be1002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.680123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-41a902dc-eef1-4b19-a95f-c33d74be1002 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:28.681340 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-41a902dc-eef1-4b19-a95f-c33d74be1002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:28.749501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:28.774096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 233/457] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:28.891458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed0b05-8bfa-4897-a0c3-18cd0b9b730c None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.892096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed0b05-8bfa-4897-a0c3-18cd0b9b730c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.892636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed0b05-8bfa-4897-a0c3-18cd0b9b730c None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.894240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 225/458] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:28.900300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a68002-7bd2-4a82-aef1-176fa11ac68f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.900861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a68002-7bd2-4a82-aef1-176fa11ac68f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.901333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a68002-7bd2-4a82-aef1-176fa11ac68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.950995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-32a68002-7bd2-4a82-aef1-176fa11ac68f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:28.952545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-32a68002-7bd2-4a82-aef1-176fa11ac68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:29.118671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f47503-9687-42ef-bb7c-8294f626f639 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.119751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f47503-9687-42ef-bb7c-8294f626f639 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.120244 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f47503-9687-42ef-bb7c-8294f626f639 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.122060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 226/459] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:29.136781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643c39-e481-41d6-a06d-6dec0cae030f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.137404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643c39-e481-41d6-a06d-6dec0cae030f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.137967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643c39-e481-41d6-a06d-6dec0cae030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.171303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ff643c39-e481-41d6-a06d-6dec0cae030f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:29.173189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ff643c39-e481-41d6-a06d-6dec0cae030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:29.181996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 234/460] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:29.194536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a68002-7bd2-4a82-aef1-176fa11ac68f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:29.450851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 227/461] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:29.510996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a68002-7bd2-4a82-aef1-176fa11ac68f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:29.563153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:29.592831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e498789-83ef-4347-a4d9-c0be6e28fb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:29.592831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e498789-83ef-4347-a4d9-c0be6e28fb84 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.593127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e498789-83ef-4347-a4d9-c0be6e28fb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.593201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e498789-83ef-4347-a4d9-c0be6e28fb84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.594955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff643c39-e481-41d6-a06d-6dec0cae030f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:29.618542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e498789-83ef-4347-a4d9-c0be6e28fb84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:29.618542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e498789-83ef-4347-a4d9-c0be6e28fb84 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-4e498789-83ef-4347-a4d9-c0be6e28fb84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:29.661428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:29.662867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:29.663738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:29.663738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:29.663738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:29.665597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 235/462] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:29.679758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e498789-83ef-4347-a4d9-c0be6e28fb84 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:29.800697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:29.819655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff643c39-e481-41d6-a06d-6dec0cae030f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:29.881505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb6932-d08a-452e-ba27-a3a590edfe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:29.882161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb6932-d08a-452e-ba27-a3a590edfe8a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.882447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb6932-d08a-452e-ba27-a3a590edfe8a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.882757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb6932-d08a-452e-ba27-a3a590edfe8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.898319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb6932-d08a-452e-ba27-a3a590edfe8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:29.900445 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb6932-d08a-452e-ba27-a3a590edfe8a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-6dbb6932-d08a-452e-ba27-a3a590edfe8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:29.922454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:29.924011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:29.925233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:29.925233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:29.925233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:29.926647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 228/463] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:29.928215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b4bec-ec56-48b3-8726-a51ed4027ac7 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.929282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b4bec-ec56-48b3-8726-a51ed4027ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.930297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b4bec-ec56-48b3-8726-a51ed4027ac7 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.936180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e498789-83ef-4347-a4d9-c0be6e28fb84 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:29.936329 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 229/464] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:29.937089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b4bec-ec56-48b3-8726-a51ed4027ac7 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:29.974454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e5a70-0461-4f3d-aedf-249eb69ad53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:29.975447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e5a70-0461-4f3d-aedf-249eb69ad53c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.975869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e5a70-0461-4f3d-aedf-249eb69ad53c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.976284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e5a70-0461-4f3d-aedf-249eb69ad53c None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.977066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e5a70-0461-4f3d-aedf-249eb69ad53c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:29.977527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e5a70-0461-4f3d-aedf-249eb69ad53c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:29.979080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e5a70-0461-4f3d-aedf-249eb69ad53c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:29.992609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 236/465] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:29.993766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e5a70-0461-4f3d-aedf-249eb69ad53c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:30.046531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:30.066996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b4bec-ec56-48b3-8726-a51ed4027ac7 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:30.113022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321d8ed-f788-4efb-b11e-e7ea03c9cfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:30.113995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321d8ed-f788-4efb-b11e-e7ea03c9cfe3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.114278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321d8ed-f788-4efb-b11e-e7ea03c9cfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.114772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321d8ed-f788-4efb-b11e-e7ea03c9cfe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.135518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:30.137562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321d8ed-f788-4efb-b11e-e7ea03c9cfe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:30.138014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e5a70-0461-4f3d-aedf-249eb69ad53c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:30.141111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321d8ed-f788-4efb-b11e-e7ea03c9cfe3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-6321d8ed-f788-4efb-b11e-e7ea03c9cfe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:30.164413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:30.165674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:30.167298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:30.167298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:30.167298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:30.167298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 230/466] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:30.168766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272728-09fc-4d01-beec-0aee60315e2a None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.168766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272728-09fc-4d01-beec-0aee60315e2a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.168766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272728-09fc-4d01-beec-0aee60315e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.194473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-48272728-09fc-4d01-beec-0aee60315e2a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:30.195725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-48272728-09fc-4d01-beec-0aee60315e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:30.195829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a499ae-df06-4949-b4e6-b5e9834bc71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:30.197755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a499ae-df06-4949-b4e6-b5e9834bc71f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.199222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a499ae-df06-4949-b4e6-b5e9834bc71f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.199222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a499ae-df06-4949-b4e6-b5e9834bc71f None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.199655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a499ae-df06-4949-b4e6-b5e9834bc71f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:30.200516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a499ae-df06-4949-b4e6-b5e9834bc71f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:30.204697 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a499ae-df06-4949-b4e6-b5e9834bc71f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:30.216890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 237/467] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:30.217568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a499ae-df06-4949-b4e6-b5e9834bc71f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:30.299594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 231/468] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:30.300508 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48272728-09fc-4d01-beec-0aee60315e2a None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:30.348352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:30.368170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a499ae-df06-4949-b4e6-b5e9834bc71f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:30.413238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd6bcf-338d-4efd-84de-892485e05bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:30.414334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd6bcf-338d-4efd-84de-892485e05bf9 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.414334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd6bcf-338d-4efd-84de-892485e05bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.414831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd6bcf-338d-4efd-84de-892485e05bf9 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.415626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd6bcf-338d-4efd-84de-892485e05bf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:30.417307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd6bcf-338d-4efd-84de-892485e05bf9 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:30.434857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48272728-09fc-4d01-beec-0aee60315e2a None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:30.470945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:30.495327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ccdd4-b313-4d62-9d7d-7191d104893e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:30.496038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ccdd4-b313-4d62-9d7d-7191d104893e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.508128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ccdd4-b313-4d62-9d7d-7191d104893e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.508128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ccdd4-b313-4d62-9d7d-7191d104893e None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.508128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ccdd4-b313-4d62-9d7d-7191d104893e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:30.508128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ccdd4-b313-4d62-9d7d-7191d104893e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:30.508128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ccdd4-b313-4d62-9d7d-7191d104893e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:30.511833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 232/469] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:30.512744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ccdd4-b313-4d62-9d7d-7191d104893e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:30.586636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 238/470] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:30:30 2022] POST /identity/v3/users => generated 246 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:30.588170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e5bbc-5eae-47df-b4d5-fe5dad11cdb7 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.588457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e5bbc-5eae-47df-b4d5-fe5dad11cdb7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.589530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e5bbc-5eae-47df-b4d5-fe5dad11cdb7 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.590590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 239/471] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:30.596729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e5bbc-5eae-47df-b4d5-fe5dad11cdb7 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:30.638848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ccdd4-b313-4d62-9d7d-7191d104893e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:30.717890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f29e6-1807-409c-b832-3c8e677b9b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:30.718566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f29e6-1807-409c-b832-3c8e677b9b75 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.719371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f29e6-1807-409c-b832-3c8e677b9b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.719806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f29e6-1807-409c-b832-3c8e677b9b75 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.720463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f29e6-1807-409c-b832-3c8e677b9b75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:30.722101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f29e6-1807-409c-b832-3c8e677b9b75 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:30.752941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e5bbc-5eae-47df-b4d5-fe5dad11cdb7 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:30.793421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48a4f-700d-4d26-82f5-0b96b1acec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:30.794039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48a4f-700d-4d26-82f5-0b96b1acec67 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.794330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48a4f-700d-4d26-82f5-0b96b1acec67 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.794659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48a4f-700d-4d26-82f5-0b96b1acec67 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.795313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee48a4f-700d-4d26-82f5-0b96b1acec67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:30.796886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee48a4f-700d-4d26-82f5-0b96b1acec67 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:30.806958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 233/472] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:30:30 2022] POST /identity/v3/users => generated 248 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:30.808504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f29e6-1807-409c-b832-3c8e677b9b75 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:30.822935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:30.892842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:30.908269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 240/473] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:30:30 2022] POST /identity/v3/users => generated 248 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:30.909607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221168d7-e1d1-49c4-ab44-66d300f71adb None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:30.909902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221168d7-e1d1-49c4-ab44-66d300f71adb None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:30.910238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221168d7-e1d1-49c4-ab44-66d300f71adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:30.966317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-221168d7-e1d1-49c4-ab44-66d300f71adb None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:30.971743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-221168d7-e1d1-49c4-ab44-66d300f71adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:30.982181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f29e6-1807-409c-b832-3c8e677b9b75 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:31.088930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850b2e4-7d8e-46d2-afcf-9e68718586bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:31.089610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850b2e4-7d8e-46d2-afcf-9e68718586bd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:31.089995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850b2e4-7d8e-46d2-afcf-9e68718586bd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:31.090200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850b2e4-7d8e-46d2-afcf-9e68718586bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:31.102937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850b2e4-7d8e-46d2-afcf-9e68718586bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:31.104551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850b2e4-7d8e-46d2-afcf-9e68718586bd None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-f850b2e4-7d8e-46d2-afcf-9e68718586bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:31.123207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:31.124201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:31.125686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:31.125686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:31.125686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:31.126640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 234/474] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:31.131090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850b2e4-7d8e-46d2-afcf-9e68718586bd None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:31.150268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 241/475] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:31.253154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:31.265191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850b2e4-7d8e-46d2-afcf-9e68718586bd None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:31.309079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5e692-14ec-4408-b783-8cafcd73cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:31.310063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5e692-14ec-4408-b783-8cafcd73cb29 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:31.310570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5e692-14ec-4408-b783-8cafcd73cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:31.311161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5e692-14ec-4408-b783-8cafcd73cb29 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:31.312011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5e692-14ec-4408-b783-8cafcd73cb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:31.312576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5e692-14ec-4408-b783-8cafcd73cb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:31.314185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5e692-14ec-4408-b783-8cafcd73cb29 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:31.326961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 235/476] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:31.335760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221168d7-e1d1-49c4-ab44-66d300f71adb None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:31.489670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5e692-14ec-4408-b783-8cafcd73cb29 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:31.533405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221168d7-e1d1-49c4-ab44-66d300f71adb None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:31.605797 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:31.623987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff901b-9230-4f5a-8703-3a291719fdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:31.624606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff901b-9230-4f5a-8703-3a291719fdcb None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:31.624937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff901b-9230-4f5a-8703-3a291719fdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:31.625392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff901b-9230-4f5a-8703-3a291719fdcb None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:31.626354 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff901b-9230-4f5a-8703-3a291719fdcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:31.629849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff901b-9230-4f5a-8703-3a291719fdcb None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:31.689519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5e692-14ec-4408-b783-8cafcd73cb29 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:31.729482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:31.730162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:31.730499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:31.730808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:31.731486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:31.732887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:31.749109 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 242/477] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:30:31 2022] POST /identity/v3/users => generated 249 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:31.799595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.sql.core [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 03:30:31.799595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 03:30:31.799595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [parameters: {'id': '4f07d83b7c3d431a960ee7e9c9c20677', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 03:30:31.799595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81942) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result.read() Aug 24 03:30:31.825512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:30:31.826921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application yield resource Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise self.value Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application session.commit() Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 03:30:31.827876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.session.flush() Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application compat.raise_( Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 03:30:31.829276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:30:31.830182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result.read() Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:30:31.845652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [parameters: {'id': '4f07d83b7c3d431a960ee7e9c9c20677', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:31.846823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 03:30:31.847869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:31.848918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 236/478] 10.222.0.82 () {60 vars in 1131 bytes} [Wed Aug 24 03:30:31 2022] POST /identity/v3/roles => generated 144 bytes in 338 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:31.848997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff901b-9230-4f5a-8703-3a291719fdcb None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:31.980619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff901b-9230-4f5a-8703-3a291719fdcb None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:32.064463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f44f0-46b4-419d-a03b-022b3961b8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:32.065116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f44f0-46b4-419d-a03b-022b3961b8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:32.065397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f44f0-46b4-419d-a03b-022b3961b8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:32.065669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f44f0-46b4-419d-a03b-022b3961b8e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:32.079148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f44f0-46b4-419d-a03b-022b3961b8e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-dd8f44f0-46b4-419d-a03b-022b3961b8e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:32.084636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:32.087856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:30:32.089864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:32.093947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 243/479] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:30:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:32.094833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:32.126939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:32.256596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928efd-e13a-4900-a24f-d58586ea0832 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:32.315076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16d49c-0d53-4a0a-8d29-1a272ef71802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:32.315634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16d49c-0d53-4a0a-8d29-1a272ef71802 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:32.315916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16d49c-0d53-4a0a-8d29-1a272ef71802 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:32.316190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16d49c-0d53-4a0a-8d29-1a272ef71802 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:32.316873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16d49c-0d53-4a0a-8d29-1a272ef71802 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:32.317233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16d49c-0d53-4a0a-8d29-1a272ef71802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:32.318647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16d49c-0d53-4a0a-8d29-1a272ef71802 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:32.337996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 237/480] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:33.084154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe55a1-3bce-412d-823f-5c5126a3a6ff None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:33.084708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe55a1-3bce-412d-823f-5c5126a3a6ff None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:33.084708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe55a1-3bce-412d-823f-5c5126a3a6ff None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:33.086332 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 244/481] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:33.094545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cff8d-8753-48d2-89a2-54172b420fae None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:33.094545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cff8d-8753-48d2-89a2-54172b420fae None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:33.094545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cff8d-8753-48d2-89a2-54172b420fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:33.143073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-0b7cff8d-8753-48d2-89a2-54172b420fae None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:33.144791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0b7cff8d-8753-48d2-89a2-54172b420fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:33.253287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 238/482] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:33.407596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc5ec9-8f5b-4522-87a8-ce479a5dccd6 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:33.407911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc5ec9-8f5b-4522-87a8-ce479a5dccd6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:33.408149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc5ec9-8f5b-4522-87a8-ce479a5dccd6 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:33.409781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 245/483] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:33.415374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32672caa-fa3d-4551-9500-b14295650ff5 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:33.415661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32672caa-fa3d-4551-9500-b14295650ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:33.415935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32672caa-fa3d-4551-9500-b14295650ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:33.429972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:33.443953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-32672caa-fa3d-4551-9500-b14295650ff5 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:33.445155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-32672caa-fa3d-4551-9500-b14295650ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:33.465836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e452fb-bcec-4a45-89a0-e7be70fa5b3b None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:33.466141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e452fb-bcec-4a45-89a0-e7be70fa5b3b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:33.466399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e452fb-bcec-4a45-89a0-e7be70fa5b3b None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:33.468215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 246/484] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:33.474788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25490985-c939-4105-a438-b4d3ff24d675 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:33.475059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25490985-c939-4105-a438-b4d3ff24d675 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:33.475334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25490985-c939-4105-a438-b4d3ff24d675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:33.504878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-25490985-c939-4105-a438-b4d3ff24d675 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:33.506084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-25490985-c939-4105-a438-b4d3ff24d675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:33.582347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:33.596105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 239/485] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:33.635242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 247/486] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:30:33.646344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32672caa-fa3d-4551-9500-b14295650ff5 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:33.825396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32672caa-fa3d-4551-9500-b14295650ff5 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:33.881432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7571cc-3674-4647-a848-bdac89ec7269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:33.882090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7571cc-3674-4647-a848-bdac89ec7269 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:33.882370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7571cc-3674-4647-a848-bdac89ec7269 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:33.882832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7571cc-3674-4647-a848-bdac89ec7269 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:33.889291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:33.903767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7571cc-3674-4647-a848-bdac89ec7269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-ad7571cc-3674-4647-a848-bdac89ec7269 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:33.909543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:33.910458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:33.911326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:33.919120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 240/487] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:33.925913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25490985-c939-4105-a438-b4d3ff24d675 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:33.986219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7571cc-3674-4647-a848-bdac89ec7269 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.088194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25490985-c939-4105-a438-b4d3ff24d675 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.120906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7571cc-3674-4647-a848-bdac89ec7269 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.135049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94dd58-75c0-45c6-8189-e55ebe5c72cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.135692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94dd58-75c0-45c6-8189-e55ebe5c72cd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.135973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94dd58-75c0-45c6-8189-e55ebe5c72cd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.136242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94dd58-75c0-45c6-8189-e55ebe5c72cd None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.136937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94dd58-75c0-45c6-8189-e55ebe5c72cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:34.137242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94dd58-75c0-45c6-8189-e55ebe5c72cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:34.138725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94dd58-75c0-45c6-8189-e55ebe5c72cd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:34.148751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 248/488] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:34.149509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94dd58-75c0-45c6-8189-e55ebe5c72cd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.164468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228458f0-05f2-4b5e-ae64-58722129ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.165099 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228458f0-05f2-4b5e-ae64-58722129ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.165421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228458f0-05f2-4b5e-ae64-58722129ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.165635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228458f0-05f2-4b5e-ae64-58722129ccf7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.177466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228458f0-05f2-4b5e-ae64-58722129ccf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-228458f0-05f2-4b5e-ae64-58722129ccf7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:34.181812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:34.182944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:34.183817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 241/489] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:34.184788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd0f07-33f0-4fa9-9657-8c6f6de57ff7 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.184892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd0f07-33f0-4fa9-9657-8c6f6de57ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.185120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd0f07-33f0-4fa9-9657-8c6f6de57ff7 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.190645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 242/490] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:34.190645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd0f07-33f0-4fa9-9657-8c6f6de57ff7 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.198932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:34.262543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:34.285538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94dd58-75c0-45c6-8189-e55ebe5c72cd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.310760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd0f07-33f0-4fa9-9657-8c6f6de57ff7 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.326105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80015e4-c5a2-4652-b54c-94403400a2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.326784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80015e4-c5a2-4652-b54c-94403400a2ff None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.327063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80015e4-c5a2-4652-b54c-94403400a2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.327590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80015e4-c5a2-4652-b54c-94403400a2ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.340485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80015e4-c5a2-4652-b54c-94403400a2ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-a80015e4-c5a2-4652-b54c-94403400a2ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:34.347149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:34.348085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:34.348881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 249/491] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:34.349553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80015e4-c5a2-4652-b54c-94403400a2ff None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.363433 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128a5e0-3627-4c98-b795-c9a6d3f0902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.364081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128a5e0-3627-4c98-b795-c9a6d3f0902a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.364362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128a5e0-3627-4c98-b795-c9a6d3f0902a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.364639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128a5e0-3627-4c98-b795-c9a6d3f0902a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.376964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128a5e0-3627-4c98-b795-c9a6d3f0902a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:34.378463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128a5e0-3627-4c98-b795-c9a6d3f0902a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-0128a5e0-3627-4c98-b795-c9a6d3f0902a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:34.394854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:34.395824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:34.402009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:34.402009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:34.402009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:34.402009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 243/492] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:34.402009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f9c08-2093-4ca6-951a-03539041f341 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.402009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f9c08-2093-4ca6-951a-03539041f341 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.402009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f9c08-2093-4ca6-951a-03539041f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.417012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-853f9c08-2093-4ca6-951a-03539041f341 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:34.418458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-853f9c08-2093-4ca6-951a-03539041f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:34.468358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:34.476484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80015e4-c5a2-4652-b54c-94403400a2ff None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.510533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 244/493] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:34.510533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f9c08-2093-4ca6-951a-03539041f341 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.538193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890ee50-0be5-473d-b1f9-5e03dc1a20ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.539224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890ee50-0be5-473d-b1f9-5e03dc1a20ff None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.539604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890ee50-0be5-473d-b1f9-5e03dc1a20ff None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.540863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890ee50-0be5-473d-b1f9-5e03dc1a20ff None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.541666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890ee50-0be5-473d-b1f9-5e03dc1a20ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:34.542073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890ee50-0be5-473d-b1f9-5e03dc1a20ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:34.543601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890ee50-0be5-473d-b1f9-5e03dc1a20ff None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:34.546857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:34.559591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 250/494] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:34.560773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890ee50-0be5-473d-b1f9-5e03dc1a20ff None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.639471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f9c08-2093-4ca6-951a-03539041f341 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.683152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890ee50-0be5-473d-b1f9-5e03dc1a20ff None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.692535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aed262-beed-4944-a29e-7aa1f4b5b896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.693712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aed262-beed-4944-a29e-7aa1f4b5b896 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.694195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aed262-beed-4944-a29e-7aa1f4b5b896 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.694711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aed262-beed-4944-a29e-7aa1f4b5b896 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.695663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aed262-beed-4944-a29e-7aa1f4b5b896 None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:34.696303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aed262-beed-4944-a29e-7aa1f4b5b896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:34.698309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aed262-beed-4944-a29e-7aa1f4b5b896 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:34.708805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 245/495] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:34.710048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aed262-beed-4944-a29e-7aa1f4b5b896 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.722286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39d71-cc6a-4ca1-b79d-24617142e0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.722936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39d71-cc6a-4ca1-b79d-24617142e0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.723220 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39d71-cc6a-4ca1-b79d-24617142e0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.723498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39d71-cc6a-4ca1-b79d-24617142e0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.724161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f39d71-cc6a-4ca1-b79d-24617142e0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:34.724936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f39d71-cc6a-4ca1-b79d-24617142e0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:34.726713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f39d71-cc6a-4ca1-b79d-24617142e0e6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:34.746937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 251/496] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:34.747882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39d71-cc6a-4ca1-b79d-24617142e0e6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.775838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:34.841790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:34.846033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aed262-beed-4944-a29e-7aa1f4b5b896 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.866828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39d71-cc6a-4ca1-b79d-24617142e0e6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:34.887541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd01bc-86de-4777-a65b-2bad434c5096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.888185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd01bc-86de-4777-a65b-2bad434c5096 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.888618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd01bc-86de-4777-a65b-2bad434c5096 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.888894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd01bc-86de-4777-a65b-2bad434c5096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.903075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd01bc-86de-4777-a65b-2bad434c5096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:34.904525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd01bc-86de-4777-a65b-2bad434c5096 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:34.906901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad8bb-7888-43b2-9613-1c53cd12c3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:34.907675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad8bb-7888-43b2-9613-1c53cd12c3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:34.908027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad8bb-7888-43b2-9613-1c53cd12c3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:34.908338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad8bb-7888-43b2-9613-1c53cd12c3e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:34.919453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad8bb-7888-43b2-9613-1c53cd12c3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:34.921011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad8bb-7888-43b2-9613-1c53cd12c3e6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-e0dd01bc-86de-4777-a65b-2bad434c5096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:34.921630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:34.928538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:34.929412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:34.929412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:34.929412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:34.929412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 246/497] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:34.929412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd01bc-86de-4777-a65b-2bad434c5096 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-f68ad8bb-7888-43b2-9613-1c53cd12c3e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:34.944475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:34.945459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:34.946318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:34.946318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:34.946318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:34.946318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 252/498] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:34.946588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad8bb-7888-43b2-9613-1c53cd12c3e6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.030366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.044272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd01bc-86de-4777-a65b-2bad434c5096 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.065262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad8bb-7888-43b2-9613-1c53cd12c3e6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.078985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed49ff-6b17-4d6f-8c86-d7fa3a9e4cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.079635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed49ff-6b17-4d6f-8c86-d7fa3a9e4cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.079910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed49ff-6b17-4d6f-8c86-d7fa3a9e4cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.080177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed49ff-6b17-4d6f-8c86-d7fa3a9e4cd4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.087776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed49ff-6b17-4d6f-8c86-d7fa3a9e4cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.089252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed49ff-6b17-4d6f-8c86-d7fa3a9e4cd4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.101607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9dcf3-76c2-4e4e-9beb-9efb59a4110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.102231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9dcf3-76c2-4e4e-9beb-9efb59a4110c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.102543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9dcf3-76c2-4e4e-9beb-9efb59a4110c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.102809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9dcf3-76c2-4e4e-9beb-9efb59a4110c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-6eed49ff-6b17-4d6f-8c86-d7fa3a9e4cd4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:35.103289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:35.104491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:35.105995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 247/499] 10.222.0.82 () {56 vars in 1093 bytes} [Wed Aug 24 03:30:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:35.107336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed49ff-6b17-4d6f-8c86-d7fa3a9e4cd4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.116477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9dcf3-76c2-4e4e-9beb-9efb59a4110c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:35.124139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-3aa9dcf3-76c2-4e4e-9beb-9efb59a4110c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:35.127013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:35.128123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:35.129139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 253/500] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:35.130546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9dcf3-76c2-4e4e-9beb-9efb59a4110c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.220444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed49ff-6b17-4d6f-8c86-d7fa3a9e4cd4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.238734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9dcf3-76c2-4e4e-9beb-9efb59a4110c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.256261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76813a51-bb8c-42e4-b874-2440154c4bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.256872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813a51-bb8c-42e4-b874-2440154c4bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.257155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813a51-bb8c-42e4-b874-2440154c4bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.257428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76813a51-bb8c-42e4-b874-2440154c4bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.258072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76813a51-bb8c-42e4-b874-2440154c4bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:35.258375 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76813a51-bb8c-42e4-b874-2440154c4bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.259908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76813a51-bb8c-42e4-b874-2440154c4bb5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.265298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.276517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 248/501] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:35.276517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7ee9-d192-472f-9f9e-4bb3e6bd6715 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.276517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7ee9-d192-472f-9f9e-4bb3e6bd6715 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.276517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7ee9-d192-472f-9f9e-4bb3e6bd6715 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.282729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30e239-21d1-42cb-a72b-7a9d97a393ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.282729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30e239-21d1-42cb-a72b-7a9d97a393ba None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.282884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 249/502] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:35.282953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30e239-21d1-42cb-a72b-7a9d97a393ba None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.282953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30e239-21d1-42cb-a72b-7a9d97a393ba None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.283114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d7ee9-d192-472f-9f9e-4bb3e6bd6715 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.283188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30e239-21d1-42cb-a72b-7a9d97a393ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:35.283188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30e239-21d1-42cb-a72b-7a9d97a393ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.283188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30e239-21d1-42cb-a72b-7a9d97a393ba None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.290768 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 254/503] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:35.291612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30e239-21d1-42cb-a72b-7a9d97a393ba None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.353064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.385801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d7ee9-d192-472f-9f9e-4bb3e6bd6715 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.395449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30e239-21d1-42cb-a72b-7a9d97a393ba None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.424626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1963b90-12fb-4761-90e1-018730e3a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.425242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1963b90-12fb-4761-90e1-018730e3a932 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.425520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1963b90-12fb-4761-90e1-018730e3a932 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.425794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1963b90-12fb-4761-90e1-018730e3a932 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.426552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1963b90-12fb-4761-90e1-018730e3a932 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:35.426862 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1963b90-12fb-4761-90e1-018730e3a932 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.428320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1963b90-12fb-4761-90e1-018730e3a932 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.431734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ec2c4-9511-4738-a858-8aa33072464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.432349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ec2c4-9511-4738-a858-8aa33072464e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.432635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ec2c4-9511-4738-a858-8aa33072464e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.432913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ec2c4-9511-4738-a858-8aa33072464e None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.433583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ec2c4-9511-4738-a858-8aa33072464e None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:35.433896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ec2c4-9511-4738-a858-8aa33072464e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.435323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ec2c4-9511-4738-a858-8aa33072464e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.443972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 255/504] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:35.444885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ec2c4-9511-4738-a858-8aa33072464e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.467146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 250/505] 10.222.0.82 () {56 vars in 1133 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:35.468553 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fda323-76ff-48e4-88da-cdf7f38242d8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.468882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fda323-76ff-48e4-88da-cdf7f38242d8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.469155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fda323-76ff-48e4-88da-cdf7f38242d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.488413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-f1fda323-76ff-48e4-88da-cdf7f38242d8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:35.489555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f1fda323-76ff-48e4-88da-cdf7f38242d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:35.503848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.555412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ec2c4-9511-4738-a858-8aa33072464e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.560221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 251/506] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:35.561069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fda323-76ff-48e4-88da-cdf7f38242d8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.604307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384446c-059f-43d6-aff2-5f45367e556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.604862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384446c-059f-43d6-aff2-5f45367e556b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.605141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384446c-059f-43d6-aff2-5f45367e556b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.605409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384446c-059f-43d6-aff2-5f45367e556b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.616163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384446c-059f-43d6-aff2-5f45367e556b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.618359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384446c-059f-43d6-aff2-5f45367e556b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-6384446c-059f-43d6-aff2-5f45367e556b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:35.639043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:35.640163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 256/507] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:35.641922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384446c-059f-43d6-aff2-5f45367e556b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.660178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.707818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fda323-76ff-48e4-88da-cdf7f38242d8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.752235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b34d68-c8fe-4966-8dd4-e27de807697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.752805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b34d68-c8fe-4966-8dd4-e27de807697a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.753092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b34d68-c8fe-4966-8dd4-e27de807697a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.753374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b34d68-c8fe-4966-8dd4-e27de807697a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.762126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b34d68-c8fe-4966-8dd4-e27de807697a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.763813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b34d68-c8fe-4966-8dd4-e27de807697a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-90b34d68-c8fe-4966-8dd4-e27de807697a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:35.778189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:35.782618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 252/508] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:35.783596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b34d68-c8fe-4966-8dd4-e27de807697a None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.790012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384446c-059f-43d6-aff2-5f45367e556b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.803071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.835032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cceea-03ca-4998-9e1e-6d11eb903c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.835909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cceea-03ca-4998-9e1e-6d11eb903c1f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.836307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cceea-03ca-4998-9e1e-6d11eb903c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.836720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cceea-03ca-4998-9e1e-6d11eb903c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:35.852489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cceea-03ca-4998-9e1e-6d11eb903c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:35.853992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cceea-03ca-4998-9e1e-6d11eb903c1f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-423cceea-03ca-4998-9e1e-6d11eb903c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:35.871012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:35.872146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:35.873296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:35.873296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:35.873296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:35.873296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 257/509] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:35.873568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cceea-03ca-4998-9e1e-6d11eb903c1f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:35.898875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b34d68-c8fe-4966-8dd4-e27de807697a None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:35.924954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:35.940926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb9858-a728-42f2-a3a0-3f758bdd9ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:35.941670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb9858-a728-42f2-a3a0-3f758bdd9ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:35.942066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb9858-a728-42f2-a3a0-3f758bdd9ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:35.942375 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb9858-a728-42f2-a3a0-3f758bdd9ca2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.001609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cceea-03ca-4998-9e1e-6d11eb903c1f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.041497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb9858-a728-42f2-a3a0-3f758bdd9ca2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.043067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb9858-a728-42f2-a3a0-3f758bdd9ca2 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.050491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584136a-1477-41bd-96b0-5d26526a8d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.051546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584136a-1477-41bd-96b0-5d26526a8d27 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.052021 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584136a-1477-41bd-96b0-5d26526a8d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.052489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584136a-1477-41bd-96b0-5d26526a8d27 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.053305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584136a-1477-41bd-96b0-5d26526a8d27 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:36.053621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584136a-1477-41bd-96b0-5d26526a8d27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.055187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584136a-1477-41bd-96b0-5d26526a8d27 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.095557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 258/510] 10.222.0.82 () {56 vars in 1137 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:36.096428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584136a-1477-41bd-96b0-5d26526a8d27 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.119898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:36.150340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 253/511] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:36.151243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb9858-a728-42f2-a3a0-3f758bdd9ca2 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.177266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:36.202195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584136a-1477-41bd-96b0-5d26526a8d27 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.238866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a29579-432a-427d-93ee-d83d7566046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.239476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a29579-432a-427d-93ee-d83d7566046a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.239756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a29579-432a-427d-93ee-d83d7566046a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.240028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a29579-432a-427d-93ee-d83d7566046a None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.240762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a29579-432a-427d-93ee-d83d7566046a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:36.241067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a29579-432a-427d-93ee-d83d7566046a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.242531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a29579-432a-427d-93ee-d83d7566046a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.258632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb9858-a728-42f2-a3a0-3f758bdd9ca2 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.259778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 259/512] 10.222.0.82 () {56 vars in 1137 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:36.260612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a29579-432a-427d-93ee-d83d7566046a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.296251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082d450-de2c-45e1-ae7c-4546a926e8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.296840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082d450-de2c-45e1-ae7c-4546a926e8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.297130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082d450-de2c-45e1-ae7c-4546a926e8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.297405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082d450-de2c-45e1-ae7c-4546a926e8e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.308131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082d450-de2c-45e1-ae7c-4546a926e8e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.309597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082d450-de2c-45e1-ae7c-4546a926e8e8 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-d082d450-de2c-45e1-ae7c-4546a926e8e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:36.325966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:36.330644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:36.331781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:36.331781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:36.331781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:36.331781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 254/513] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:36.331781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082d450-de2c-45e1-ae7c-4546a926e8e8 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.352646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:36.375084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a29579-432a-427d-93ee-d83d7566046a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.409857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:36.411860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691b5f5-1fa9-4a1b-b3d1-90949283dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.412467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691b5f5-1fa9-4a1b-b3d1-90949283dc52 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.412745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691b5f5-1fa9-4a1b-b3d1-90949283dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.413016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691b5f5-1fa9-4a1b-b3d1-90949283dc52 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.413659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691b5f5-1fa9-4a1b-b3d1-90949283dc52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:36.413968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691b5f5-1fa9-4a1b-b3d1-90949283dc52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.415432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691b5f5-1fa9-4a1b-b3d1-90949283dc52 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.426743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 260/514] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:36.427562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691b5f5-1fa9-4a1b-b3d1-90949283dc52 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.440589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082d450-de2c-45e1-ae7c-4546a926e8e8 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.476893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66f730-ceae-40a3-b150-ac9e3d5cdf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.477509 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66f730-ceae-40a3-b150-ac9e3d5cdf50 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.477793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66f730-ceae-40a3-b150-ac9e3d5cdf50 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.478104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66f730-ceae-40a3-b150-ac9e3d5cdf50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.533684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691b5f5-1fa9-4a1b-b3d1-90949283dc52 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.550010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66f730-ceae-40a3-b150-ac9e3d5cdf50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.551504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66f730-ceae-40a3-b150-ac9e3d5cdf50 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.567616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3196589-9b97-443f-a22b-39c4c51c95fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.568243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3196589-9b97-443f-a22b-39c4c51c95fd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.568524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3196589-9b97-443f-a22b-39c4c51c95fd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.568799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3196589-9b97-443f-a22b-39c4c51c95fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.575925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:36.580594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3196589-9b97-443f-a22b-39c4c51c95fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.582095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3196589-9b97-443f-a22b-39c4c51c95fd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-f3196589-9b97-443f-a22b-39c4c51c95fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:36.597410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:36.598659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:36.600195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:36.600195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:36.600195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:36.600195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 261/515] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:36.600195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3196589-9b97-443f-a22b-39c4c51c95fd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.630289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:36.660944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 255/516] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:36.662086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66f730-ceae-40a3-b150-ac9e3d5cdf50 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.710903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3196589-9b97-443f-a22b-39c4c51c95fd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.746123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd55a3-385c-41b2-bd09-96413fb06c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.746774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd55a3-385c-41b2-bd09-96413fb06c24 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.747057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd55a3-385c-41b2-bd09-96413fb06c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.747331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd55a3-385c-41b2-bd09-96413fb06c24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.764615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66f730-ceae-40a3-b150-ac9e3d5cdf50 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.800655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fc6c7-24bb-4c8b-8b20-a8f3c807e2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.801187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fc6c7-24bb-4c8b-8b20-a8f3c807e2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.801468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fc6c7-24bb-4c8b-8b20-a8f3c807e2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.801747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fc6c7-24bb-4c8b-8b20-a8f3c807e2b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.802481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fc6c7-24bb-4c8b-8b20-a8f3c807e2b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:36.802826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fc6c7-24bb-4c8b-8b20-a8f3c807e2b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.804215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fc6c7-24bb-4c8b-8b20-a8f3c807e2b6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.808591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:36.817336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 256/517] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:36.823077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fc6c7-24bb-4c8b-8b20-a8f3c807e2b6 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.823201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd55a3-385c-41b2-bd09-96413fb06c24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.823201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd55a3-385c-41b2-bd09-96413fb06c24 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.853445 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:36.926997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fc6c7-24bb-4c8b-8b20-a8f3c807e2b6 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:36.927593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 262/518] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:36.928439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd55a3-385c-41b2-bd09-96413fb06c24 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:36.966195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efff6f-589f-4d27-8451-66bf7350603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:36.966866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efff6f-589f-4d27-8451-66bf7350603f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:36.967147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efff6f-589f-4d27-8451-66bf7350603f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:36.967415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efff6f-589f-4d27-8451-66bf7350603f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:36.975130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efff6f-589f-4d27-8451-66bf7350603f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:36.976581 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efff6f-589f-4d27-8451-66bf7350603f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-71efff6f-589f-4d27-8451-66bf7350603f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:36.988763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:36.989892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 257/519] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:36.990935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efff6f-589f-4d27-8451-66bf7350603f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.027954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.038596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd55a3-385c-41b2-bd09-96413fb06c24 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.074156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3b93b-b25e-40dd-8c05-dca46ef5863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.074709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3b93b-b25e-40dd-8c05-dca46ef5863c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.074988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3b93b-b25e-40dd-8c05-dca46ef5863c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.075258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3b93b-b25e-40dd-8c05-dca46ef5863c None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.075896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3b93b-b25e-40dd-8c05-dca46ef5863c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:37.076198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3b93b-b25e-40dd-8c05-dca46ef5863c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.077568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3b93b-b25e-40dd-8c05-dca46ef5863c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.088802 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 263/520] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:37.089587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3b93b-b25e-40dd-8c05-dca46ef5863c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.098707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.105824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efff6f-589f-4d27-8451-66bf7350603f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.148351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b43b81-a3ef-423e-b8d7-ecb6254744cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.149284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b43b81-a3ef-423e-b8d7-ecb6254744cf None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.149615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b43b81-a3ef-423e-b8d7-ecb6254744cf None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.150116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b43b81-a3ef-423e-b8d7-ecb6254744cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.162030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b43b81-a3ef-423e-b8d7-ecb6254744cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.163532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b43b81-a3ef-423e-b8d7-ecb6254744cf None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-51b43b81-a3ef-423e-b8d7-ecb6254744cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:37.178848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:37.179957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:37.180809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:37.180809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:37.180809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:37.181308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 258/521] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:37.182127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b43b81-a3ef-423e-b8d7-ecb6254744cf None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.197542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3b93b-b25e-40dd-8c05-dca46ef5863c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.234101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83504c18-c22a-418f-9b32-da10288bc183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.234101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83504c18-c22a-418f-9b32-da10288bc183 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.234101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83504c18-c22a-418f-9b32-da10288bc183 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.235995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83504c18-c22a-418f-9b32-da10288bc183 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.235995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83504c18-c22a-418f-9b32-da10288bc183 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.237583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83504c18-c22a-418f-9b32-da10288bc183 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.282565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b43b81-a3ef-423e-b8d7-ecb6254744cf None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.315990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7029c7-0a72-41c7-b303-c0350ceaff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.316699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7029c7-0a72-41c7-b303-c0350ceaff26 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.317088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7029c7-0a72-41c7-b303-c0350ceaff26 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.317413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7029c7-0a72-41c7-b303-c0350ceaff26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.327974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7029c7-0a72-41c7-b303-c0350ceaff26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.329427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7029c7-0a72-41c7-b303-c0350ceaff26 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.332006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.342901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 264/522] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:30:37 2022] POST /identity/v3/users => generated 247 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-ae7029c7-0a72-41c7-b303-c0350ceaff26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:37.344462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:37.345395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:37.346357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:37.346357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:37.346357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:37.346847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83504c18-c22a-418f-9b32-da10288bc183 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.346933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 259/523] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:37.346933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7029c7-0a72-41c7-b303-c0350ceaff26 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.364173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.476431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83504c18-c22a-418f-9b32-da10288bc183 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.480117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7029c7-0a72-41c7-b303-c0350ceaff26 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.520076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672005c2-7bfa-4794-8c71-a261198f7714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.520702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672005c2-7bfa-4794-8c71-a261198f7714 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.520989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672005c2-7bfa-4794-8c71-a261198f7714 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.521951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672005c2-7bfa-4794-8c71-a261198f7714 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.522054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672005c2-7bfa-4794-8c71-a261198f7714 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:37.522305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672005c2-7bfa-4794-8c71-a261198f7714 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.523798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672005c2-7bfa-4794-8c71-a261198f7714 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.525616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7d491-79a5-4835-b203-7f81d1bb0166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.526260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7d491-79a5-4835-b203-7f81d1bb0166 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.526656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7d491-79a5-4835-b203-7f81d1bb0166 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.526923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7d491-79a5-4835-b203-7f81d1bb0166 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.536318 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7d491-79a5-4835-b203-7f81d1bb0166 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.537162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7d491-79a5-4835-b203-7f81d1bb0166 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.545315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 265/524] 10.222.0.82 () {56 vars in 1139 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:37.546192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672005c2-7bfa-4794-8c71-a261198f7714 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-93b7d491-79a5-4835-b203-7f81d1bb0166 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:37.553226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:37.554283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 260/525] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:37.555395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7d491-79a5-4835-b203-7f81d1bb0166 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.619558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.625296 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.660302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672005c2-7bfa-4794-8c71-a261198f7714 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.679073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7d491-79a5-4835-b203-7f81d1bb0166 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.700001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa72bc-3be1-4b88-938c-493133cb5e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.700626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa72bc-3be1-4b88-938c-493133cb5e78 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.700913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa72bc-3be1-4b88-938c-493133cb5e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.701191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa72bc-3be1-4b88-938c-493133cb5e78 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.701853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa72bc-3be1-4b88-938c-493133cb5e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:37.702161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa72bc-3be1-4b88-938c-493133cb5e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.704259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa72bc-3be1-4b88-938c-493133cb5e78 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.716165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 266/526] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:37.717157 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa72bc-3be1-4b88-938c-493133cb5e78 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.727083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85811d-969e-4025-b29b-9e53ca893f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.728465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85811d-969e-4025-b29b-9e53ca893f6c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.729000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85811d-969e-4025-b29b-9e53ca893f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.729565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85811d-969e-4025-b29b-9e53ca893f6c None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.731223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85811d-969e-4025-b29b-9e53ca893f6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:37.731785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85811d-969e-4025-b29b-9e53ca893f6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.733500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85811d-969e-4025-b29b-9e53ca893f6c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.749309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 261/527] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:37.751495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85811d-969e-4025-b29b-9e53ca893f6c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.852247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa72bc-3be1-4b88-938c-493133cb5e78 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.867200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85811d-969e-4025-b29b-9e53ca893f6c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:37.877517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.887461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:37.890363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c32d-cf75-4bdd-9d6c-f082c9cabae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.891546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c32d-cf75-4bdd-9d6c-f082c9cabae5 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.893306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c32d-cf75-4bdd-9d6c-f082c9cabae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.893306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c32d-cf75-4bdd-9d6c-f082c9cabae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.916926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f7b90-a93f-40bb-9979-54f2cf6c4112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:37.917551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f7b90-a93f-40bb-9979-54f2cf6c4112 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:37.922384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f7b90-a93f-40bb-9979-54f2cf6c4112 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:37.922384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f7b90-a93f-40bb-9979-54f2cf6c4112 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:37.922384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f7b90-a93f-40bb-9979-54f2cf6c4112 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:37.922384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f7b90-a93f-40bb-9979-54f2cf6c4112 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.922384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f7b90-a93f-40bb-9979-54f2cf6c4112 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:37.946568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 262/528] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:37.946568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f7b90-a93f-40bb-9979-54f2cf6c4112 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:37.978385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1c32d-cf75-4bdd-9d6c-f082c9cabae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:37.980169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1c32d-cf75-4bdd-9d6c-f082c9cabae5 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.069459 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f7b90-a93f-40bb-9979-54f2cf6c4112 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.104460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 267/529] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:38.105710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c32d-cf75-4bdd-9d6c-f082c9cabae5 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.116004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e9105-a6a4-44d0-b773-5f02555208d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.117000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e9105-a6a4-44d0-b773-5f02555208d3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.117287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e9105-a6a4-44d0-b773-5f02555208d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.117566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e9105-a6a4-44d0-b773-5f02555208d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.125495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e9105-a6a4-44d0-b773-5f02555208d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.127310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e9105-a6a4-44d0-b773-5f02555208d3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.137434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-7d7e9105-a6a4-44d0-b773-5f02555208d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:38.138689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:38.139593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 263/530] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:38.140701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e9105-a6a4-44d0-b773-5f02555208d3 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.183006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:38.227069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c32d-cf75-4bdd-9d6c-f082c9cabae5 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.262471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e9105-a6a4-44d0-b773-5f02555208d3 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.267518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e15ea-dbf3-4832-b9e2-c94c5a5c57af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.268158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e15ea-dbf3-4832-b9e2-c94c5a5c57af None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.268443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e15ea-dbf3-4832-b9e2-c94c5a5c57af None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.268723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e15ea-dbf3-4832-b9e2-c94c5a5c57af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.276030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e15ea-dbf3-4832-b9e2-c94c5a5c57af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.277747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e15ea-dbf3-4832-b9e2-c94c5a5c57af None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-083e15ea-dbf3-4832-b9e2-c94c5a5c57af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:38.290015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:38.290956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:38.291820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:38.291820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:38.291820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:38.291820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:38.291820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:38.291820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:38.291820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:38.291820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:38.296056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 268/531] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:38.297081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e15ea-dbf3-4832-b9e2-c94c5a5c57af None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.303647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa3f47-a311-4593-9184-03c711cdcf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.304261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa3f47-a311-4593-9184-03c711cdcf5e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.304566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa3f47-a311-4593-9184-03c711cdcf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.304876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa3f47-a311-4593-9184-03c711cdcf5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.305548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa3f47-a311-4593-9184-03c711cdcf5e None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=0f34fe3fe912498b90dbf950e7fc999d) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:38.305859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa3f47-a311-4593-9184-03c711cdcf5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.307369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa3f47-a311-4593-9184-03c711cdcf5e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.314150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 264/532] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/role_assignments?user.id=0f34fe3fe912498b90dbf950e7fc999d&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:38.315004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa3f47-a311-4593-9184-03c711cdcf5e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.443360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:38.450981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:38.455147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa3f47-a311-4593-9184-03c711cdcf5e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.467057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e15ea-dbf3-4832-b9e2-c94c5a5c57af None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.503952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c434588-5006-4db0-8a56-5cc8277801ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.504983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c434588-5006-4db0-8a56-5cc8277801ef None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.505480 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c434588-5006-4db0-8a56-5cc8277801ef None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.506039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c434588-5006-4db0-8a56-5cc8277801ef None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.507098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c434588-5006-4db0-8a56-5cc8277801ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:38.507591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c434588-5006-4db0-8a56-5cc8277801ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.512405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c434588-5006-4db0-8a56-5cc8277801ef None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.515631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a40eb-c0a5-4a3e-95d6-4d39010a6f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.516333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a40eb-c0a5-4a3e-95d6-4d39010a6f73 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.516802 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a40eb-c0a5-4a3e-95d6-4d39010a6f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.517089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a40eb-c0a5-4a3e-95d6-4d39010a6f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.528066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a40eb-c0a5-4a3e-95d6-4d39010a6f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.529111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 265/533] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:38.529564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a40eb-c0a5-4a3e-95d6-4d39010a6f73 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.530673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c434588-5006-4db0-8a56-5cc8277801ef None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-789a40eb-c0a5-4a3e-95d6-4d39010a6f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:38.547431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:38.548605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:38.549546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:38.549546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:38.549546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:38.549546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 269/534] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:38.556725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a40eb-c0a5-4a3e-95d6-4d39010a6f73 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.651159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c434588-5006-4db0-8a56-5cc8277801ef None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.669083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a40eb-c0a5-4a3e-95d6-4d39010a6f73 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.698116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf53387-4d3c-44d0-9516-7fa06201f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.698689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf53387-4d3c-44d0-9516-7fa06201f002 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.698976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf53387-4d3c-44d0-9516-7fa06201f002 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.699454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf53387-4d3c-44d0-9516-7fa06201f002 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.700263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf53387-4d3c-44d0-9516-7fa06201f002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:38.700579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf53387-4d3c-44d0-9516-7fa06201f002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.702188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf53387-4d3c-44d0-9516-7fa06201f002 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.705031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad0d69-fbbc-40f5-b505-7e1b62667fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.705709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad0d69-fbbc-40f5-b505-7e1b62667fae None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.706000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad0d69-fbbc-40f5-b505-7e1b62667fae None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.706282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad0d69-fbbc-40f5-b505-7e1b62667fae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.707063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad0d69-fbbc-40f5-b505-7e1b62667fae None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=c89caed732274718aeaa0d8ffeef571b) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:38.707430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad0d69-fbbc-40f5-b505-7e1b62667fae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.708955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad0d69-fbbc-40f5-b505-7e1b62667fae None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.712134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:38.715747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 270/535] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/role_assignments?user.id=c89caed732274718aeaa0d8ffeef571b&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:38.716559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad0d69-fbbc-40f5-b505-7e1b62667fae None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.717642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 266/536] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:38.727608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf53387-4d3c-44d0-9516-7fa06201f002 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.798906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:38.842545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad0d69-fbbc-40f5-b505-7e1b62667fae None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:38.919230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c00bf1-c83a-4682-b81c-8beb9b7d659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:38.919851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c00bf1-c83a-4682-b81c-8beb9b7d659b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:38.922538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c00bf1-c83a-4682-b81c-8beb9b7d659b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:38.922538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c00bf1-c83a-4682-b81c-8beb9b7d659b None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:38.922538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c00bf1-c83a-4682-b81c-8beb9b7d659b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:38.922538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c00bf1-c83a-4682-b81c-8beb9b7d659b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:38.923360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c00bf1-c83a-4682-b81c-8beb9b7d659b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:38.968510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 271/537] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:38.970541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c00bf1-c83a-4682-b81c-8beb9b7d659b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:38.999216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf53387-4d3c-44d0-9516-7fa06201f002 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.076534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:39.118028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b0d8c-4d53-42fa-9e10-864b35eb247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.120188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b0d8c-4d53-42fa-9e10-864b35eb247e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.120188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b0d8c-4d53-42fa-9e10-864b35eb247e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.120188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b0d8c-4d53-42fa-9e10-864b35eb247e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.120188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b0d8c-4d53-42fa-9e10-864b35eb247e None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=4ff3f53ddbb04eb8916b9ccfdb03dd80) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:39.120496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b0d8c-4d53-42fa-9e10-864b35eb247e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.121790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b0d8c-4d53-42fa-9e10-864b35eb247e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.132143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c00bf1-c83a-4682-b81c-8beb9b7d659b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.146628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 267/538] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/role_assignments?user.id=4ff3f53ddbb04eb8916b9ccfdb03dd80&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 259 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:39.205700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e7dfe-8261-4a58-9bcf-c76f62e34f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.206342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e7dfe-8261-4a58-9bcf-c76f62e34f06 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.211105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e7dfe-8261-4a58-9bcf-c76f62e34f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.211706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e7dfe-8261-4a58-9bcf-c76f62e34f06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.228737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e7dfe-8261-4a58-9bcf-c76f62e34f06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.230156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e7dfe-8261-4a58-9bcf-c76f62e34f06 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-e24e7dfe-8261-4a58-9bcf-c76f62e34f06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:39.248907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:39.249828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:39.250929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:39.250929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:39.250929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:39.250929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:39.250929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:39.250929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:39.250929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:39.250929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:39.259165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 272/539] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:39.265103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b0d8c-4d53-42fa-9e10-864b35eb247e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.417545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173a4ab-6fc3-4647-9ce2-c0a8ade875ff None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.417819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173a4ab-6fc3-4647-9ce2-c0a8ade875ff None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.418102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173a4ab-6fc3-4647-9ce2-c0a8ade875ff None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.419288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:39.419759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 273/540] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:39.426162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3051c2-0e04-42ec-8b6f-36e67b995b61 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.426494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3051c2-0e04-42ec-8b6f-36e67b995b61 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.426794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3051c2-0e04-42ec-8b6f-36e67b995b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.442031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b0d8c-4d53-42fa-9e10-864b35eb247e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.447203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6f3051c2-0e04-42ec-8b6f-36e67b995b61 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:39.449429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6f3051c2-0e04-42ec-8b6f-36e67b995b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:39.521259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0fdb3-b070-4a4f-b323-128da2172f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.521823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0fdb3-b070-4a4f-b323-128da2172f87 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.522195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0fdb3-b070-4a4f-b323-128da2172f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.522473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0fdb3-b070-4a4f-b323-128da2172f87 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.523270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0fdb3-b070-4a4f-b323-128da2172f87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:39.523580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0fdb3-b070-4a4f-b323-128da2172f87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.525058 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0fdb3-b070-4a4f-b323-128da2172f87 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.551055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 268/541] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:39.557024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0fdb3-b070-4a4f-b323-128da2172f87 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:39.580848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 274/542] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:39.685403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0fdb3-b070-4a4f-b323-128da2172f87 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:39.757946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0667d12-87ce-4d5e-8c77-fdad2f1f8d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:39.758682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667d12-87ce-4d5e-8c77-fdad2f1f8d39 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:39.758972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667d12-87ce-4d5e-8c77-fdad2f1f8d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:39.759250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667d12-87ce-4d5e-8c77-fdad2f1f8d39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:39.759991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667d12-87ce-4d5e-8c77-fdad2f1f8d39 None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=f81cc77e85f84d358471dd9efab45cd4) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:39.760302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667d12-87ce-4d5e-8c77-fdad2f1f8d39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:39.761765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667d12-87ce-4d5e-8c77-fdad2f1f8d39 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:39.775271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 269/543] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/role_assignments?user.id=f81cc77e85f84d358471dd9efab45cd4&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:39.951811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3051c2-0e04-42ec-8b6f-36e67b995b61 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:40.034359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:40.135775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3051c2-0e04-42ec-8b6f-36e67b995b61 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:40.239203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d6b73-a5b3-407f-b177-4db59d823839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:40.240551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d6b73-a5b3-407f-b177-4db59d823839 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.240978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d6b73-a5b3-407f-b177-4db59d823839 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.241392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d6b73-a5b3-407f-b177-4db59d823839 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.266897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d6b73-a5b3-407f-b177-4db59d823839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-5b9d6b73-a5b3-407f-b177-4db59d823839 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:40.271927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:40.273026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:40.273843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:40.283009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 275/544] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:40.288628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0667d12-87ce-4d5e-8c77-fdad2f1f8d39 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:40.326888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:40.464750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0667d12-87ce-4d5e-8c77-fdad2f1f8d39 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:40.516781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85961f72-042d-49fd-bae1-09e718127b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:40.517415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85961f72-042d-49fd-bae1-09e718127b5c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.517694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85961f72-042d-49fd-bae1-09e718127b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.517971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85961f72-042d-49fd-bae1-09e718127b5c None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.518670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85961f72-042d-49fd-bae1-09e718127b5c None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:40.518974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85961f72-042d-49fd-bae1-09e718127b5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:40.520350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85961f72-042d-49fd-bae1-09e718127b5c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:40.530771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 270/545] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:40.538265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d6b73-a5b3-407f-b177-4db59d823839 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:40.540438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1d5b8-23bd-476f-a440-dd91a4e90c6e None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.540669 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1d5b8-23bd-476f-a440-dd91a4e90c6e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.540955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1d5b8-23bd-476f-a440-dd91a4e90c6e None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.542478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 271/546] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:40.549358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138d516-5555-4b71-9164-62f4df37ac9e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.550008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138d516-5555-4b71-9164-62f4df37ac9e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.550493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138d516-5555-4b71-9164-62f4df37ac9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.586813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6138d516-5555-4b71-9164-62f4df37ac9e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:40.588080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6138d516-5555-4b71-9164-62f4df37ac9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:40.714739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d6b73-a5b3-407f-b177-4db59d823839 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:40.728709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:40.763613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7de9d3-6652-4d4f-9596-3f88021ca96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:40.764678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7de9d3-6652-4d4f-9596-3f88021ca96c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:40.765191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7de9d3-6652-4d4f-9596-3f88021ca96c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:40.765695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7de9d3-6652-4d4f-9596-3f88021ca96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:40.776676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:40.784985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 272/547] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:30:40.791401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7de9d3-6652-4d4f-9596-3f88021ca96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:40.792845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7de9d3-6652-4d4f-9596-3f88021ca96c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-fb7de9d3-6652-4d4f-9596-3f88021ca96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:40.809436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:40.810458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:40.811902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:40.811902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:40.811902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:40.811902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 276/548] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:40.826143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138d516-5555-4b71-9164-62f4df37ac9e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.115399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138d516-5555-4b71-9164-62f4df37ac9e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.221878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd066f1-4aea-4f53-b51e-c1bbb8df3638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.223695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd066f1-4aea-4f53-b51e-c1bbb8df3638 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.224495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd066f1-4aea-4f53-b51e-c1bbb8df3638 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.225290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd066f1-4aea-4f53-b51e-c1bbb8df3638 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.226832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd066f1-4aea-4f53-b51e-c1bbb8df3638 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:41.227661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd066f1-4aea-4f53-b51e-c1bbb8df3638 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:41.230586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd066f1-4aea-4f53-b51e-c1bbb8df3638 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:41.265233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 273/549] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:41.273005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7de9d3-6652-4d4f-9596-3f88021ca96c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.284923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd066f1-4aea-4f53-b51e-c1bbb8df3638 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.401044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:41.479735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd066f1-4aea-4f53-b51e-c1bbb8df3638 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.494242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7de9d3-6652-4d4f-9596-3f88021ca96c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.546693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2945cc0-4c61-428f-8cc3-c7117a7b7517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.548801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2945cc0-4c61-428f-8cc3-c7117a7b7517 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.549549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2945cc0-4c61-428f-8cc3-c7117a7b7517 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.550346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2945cc0-4c61-428f-8cc3-c7117a7b7517 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.570401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175d1d2-634a-4f8d-9390-efb844613948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.571438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175d1d2-634a-4f8d-9390-efb844613948 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.571838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2945cc0-4c61-428f-8cc3-c7117a7b7517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:41.571999 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175d1d2-634a-4f8d-9390-efb844613948 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.572338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175d1d2-634a-4f8d-9390-efb844613948 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-e2945cc0-4c61-428f-8cc3-c7117a7b7517 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:41.582416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:41.584251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:41.585949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:41.589854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175d1d2-634a-4f8d-9390-efb844613948 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:41.591290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175d1d2-634a-4f8d-9390-efb844613948 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:41.598879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 274/550] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:41.601681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e549d1-7713-4446-a11e-d9f88a21e196 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.602536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e549d1-7713-4446-a11e-d9f88a21e196 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.603300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e549d1-7713-4446-a11e-d9f88a21e196 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.606311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 275/551] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:41.611697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e549d1-7713-4446-a11e-d9f88a21e196 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-5175d1d2-634a-4f8d-9390-efb844613948 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:41.617689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:41.619502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:30:41.621276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:41.625921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 277/552] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:30:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:41.636936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dce551-0b59-4c75-822b-be05b0d28ed0 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.637386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dce551-0b59-4c75-822b-be05b0d28ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.637847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dce551-0b59-4c75-822b-be05b0d28ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.668726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-20dce551-0b59-4c75-822b-be05b0d28ed0 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:41.670712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-20dce551-0b59-4c75-822b-be05b0d28ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:41.705366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:41.818099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 278/553] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:41.819570 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dce551-0b59-4c75-822b-be05b0d28ed0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.819900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e549d1-7713-4446-a11e-d9f88a21e196 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:41.857410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:41.884234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf5d0a-3638-4ec3-94c6-09586f1a4e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:41.885720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf5d0a-3638-4ec3-94c6-09586f1a4e81 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.886496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf5d0a-3638-4ec3-94c6-09586f1a4e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.887238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf5d0a-3638-4ec3-94c6-09586f1a4e81 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.888713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf5d0a-3638-4ec3-94c6-09586f1a4e81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:41.889574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf5d0a-3638-4ec3-94c6-09586f1a4e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:41.892405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf5d0a-3638-4ec3-94c6-09586f1a4e81 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:41.910247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 276/554] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:41.918156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24730dac-654c-46c4-b126-08a44dfffbe0 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:41.924977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24730dac-654c-46c4-b126-08a44dfffbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:41.924977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24730dac-654c-46c4-b126-08a44dfffbe0 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:41.924977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 277/555] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:41.927542 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24730dac-654c-46c4-b126-08a44dfffbe0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:41.992312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dce551-0b59-4c75-822b-be05b0d28ed0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.035372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eb27f-f46b-4eae-aac4-5099ef2a5559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.036006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eb27f-f46b-4eae-aac4-5099ef2a5559 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.036286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eb27f-f46b-4eae-aac4-5099ef2a5559 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.036588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eb27f-f46b-4eae-aac4-5099ef2a5559 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.037353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eb27f-f46b-4eae-aac4-5099ef2a5559 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:42.037656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eb27f-f46b-4eae-aac4-5099ef2a5559 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.039643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eb27f-f46b-4eae-aac4-5099ef2a5559 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.067287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24730dac-654c-46c4-b126-08a44dfffbe0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.070860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 279/556] 10.222.0.82 () {56 vars in 1135 bytes} [Wed Aug 24 03:30:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:42.072549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4284-01fc-481d-9eda-dea68236787e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.074948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4284-01fc-481d-9eda-dea68236787e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.074948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4284-01fc-481d-9eda-dea68236787e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.107993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-358c4284-01fc-481d-9eda-dea68236787e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:42.110269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-358c4284-01fc-481d-9eda-dea68236787e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:42.115514 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:42.129430 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f37159-b29d-42ac-9b20-2a863081164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.130440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f37159-b29d-42ac-9b20-2a863081164b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.130957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f37159-b29d-42ac-9b20-2a863081164b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.140396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f37159-b29d-42ac-9b20-2a863081164b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.146583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f37159-b29d-42ac-9b20-2a863081164b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.148663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f37159-b29d-42ac-9b20-2a863081164b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-a3f37159-b29d-42ac-9b20-2a863081164b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:42.167229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:42.168319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:42.169195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:42.169195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:42.169195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:42.170676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 278/557] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:42.171766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f37159-b29d-42ac-9b20-2a863081164b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.202646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 280/558] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:42.202646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4284-01fc-481d-9eda-dea68236787e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.222017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:42.312826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f37159-b29d-42ac-9b20-2a863081164b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.336266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4284-01fc-481d-9eda-dea68236787e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.357280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cb2c6-e998-4b47-855a-c1c6ef0ef27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.358245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb2c6-e998-4b47-855a-c1c6ef0ef27c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.358490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb2c6-e998-4b47-855a-c1c6ef0ef27c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.358796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb2c6-e998-4b47-855a-c1c6ef0ef27c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.376034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cc697-2bb5-44a4-8638-fbfbd4859ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.376687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cc697-2bb5-44a4-8638-fbfbd4859ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.376969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cc697-2bb5-44a4-8638-fbfbd4859ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.377246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cc697-2bb5-44a4-8638-fbfbd4859ff1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.386712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:42.395253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747cc697-2bb5-44a4-8638-fbfbd4859ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-747cc697-2bb5-44a4-8638-fbfbd4859ff1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:42.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:42.409800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 281/559] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:42.410792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cc697-2bb5-44a4-8638-fbfbd4859ff1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.460529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cb2c6-e998-4b47-855a-c1c6ef0ef27c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.461945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cb2c6-e998-4b47-855a-c1c6ef0ef27c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.502584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:42.543901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cc697-2bb5-44a4-8638-fbfbd4859ff1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.580745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce63c9a-8eee-4a20-908b-71cda5c19a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.581366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce63c9a-8eee-4a20-908b-71cda5c19a05 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.581643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce63c9a-8eee-4a20-908b-71cda5c19a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.581920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce63c9a-8eee-4a20-908b-71cda5c19a05 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.582606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce63c9a-8eee-4a20-908b-71cda5c19a05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:42.582914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce63c9a-8eee-4a20-908b-71cda5c19a05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.584313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce63c9a-8eee-4a20-908b-71cda5c19a05 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.588436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 279/560] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:42.589727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5fe7f-3f2a-4240-9769-1bca45c48b45 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.590016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5fe7f-3f2a-4240-9769-1bca45c48b45 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.590292 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5fe7f-3f2a-4240-9769-1bca45c48b45 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.591708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 280/561] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:42.592417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5fe7f-3f2a-4240-9769-1bca45c48b45 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.597598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 282/562] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:42.598541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce63c9a-8eee-4a20-908b-71cda5c19a05 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.671486 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:42.711965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5fe7f-3f2a-4240-9769-1bca45c48b45 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.715399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce63c9a-8eee-4a20-908b-71cda5c19a05 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.747554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e65907-d7af-4b76-83ce-61b87ba7edc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.748176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e65907-d7af-4b76-83ce-61b87ba7edc4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.748456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e65907-d7af-4b76-83ce-61b87ba7edc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.748784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e65907-d7af-4b76-83ce-61b87ba7edc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.749466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e65907-d7af-4b76-83ce-61b87ba7edc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:42.749775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e65907-d7af-4b76-83ce-61b87ba7edc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.751881 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e65907-d7af-4b76-83ce-61b87ba7edc4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.753484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:42.755001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36b7cb-654b-440c-aff5-22891e7e0b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.755741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36b7cb-654b-440c-aff5-22891e7e0b2b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.756034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36b7cb-654b-440c-aff5-22891e7e0b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.756306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36b7cb-654b-440c-aff5-22891e7e0b2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.761631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 281/563] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:42.762831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e65907-d7af-4b76-83ce-61b87ba7edc4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.768056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36b7cb-654b-440c-aff5-22891e7e0b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-ce36b7cb-654b-440c-aff5-22891e7e0b2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:42.773361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:42.779430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 283/564] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:42.780267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca0ab6-d425-4c64-9fad-989d8b8bfdbe None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.781131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca0ab6-d425-4c64-9fad-989d8b8bfdbe None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.781131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca0ab6-d425-4c64-9fad-989d8b8bfdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.795096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-35ca0ab6-d425-4c64-9fad-989d8b8bfdbe None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:42.796326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-35ca0ab6-d425-4c64-9fad-989d8b8bfdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:42.878895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e65907-d7af-4b76-83ce-61b87ba7edc4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:42.886752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 284/565] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:42.887445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca0ab6-d425-4c64-9fad-989d8b8bfdbe None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:42.920472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc440f08-ffc1-45ab-9b99-9f58aaf6700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:42.921121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc440f08-ffc1-45ab-9b99-9f58aaf6700d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:42.928885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc440f08-ffc1-45ab-9b99-9f58aaf6700d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:42.928885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc440f08-ffc1-45ab-9b99-9f58aaf6700d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:42.934031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc440f08-ffc1-45ab-9b99-9f58aaf6700d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:42.935531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc440f08-ffc1-45ab-9b99-9f58aaf6700d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:42.946970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-cc440f08-ffc1-45ab-9b99-9f58aaf6700d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:42.954868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:42.956281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:42.957321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:42.957321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:42.957321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:42.961671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 282/566] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:42.962573 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc440f08-ffc1-45ab-9b99-9f58aaf6700d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.017652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca0ab6-d425-4c64-9fad-989d8b8bfdbe None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.040570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:43.074910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e6759-23b3-4b99-9c94-95bc622b74e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.076288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e6759-23b3-4b99-9c94-95bc622b74e6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.076288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e6759-23b3-4b99-9c94-95bc622b74e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.076571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e6759-23b3-4b99-9c94-95bc622b74e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.089895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e6759-23b3-4b99-9c94-95bc622b74e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.092465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e6759-23b3-4b99-9c94-95bc622b74e6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-064e6759-23b3-4b99-9c94-95bc622b74e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:43.114927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:43.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:43.117703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 285/567] 10.222.0.82 () {56 vars in 1093 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:43.118490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e6759-23b3-4b99-9c94-95bc622b74e6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.140540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc440f08-ffc1-45ab-9b99-9f58aaf6700d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.205166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a80aa-17af-4303-b3f4-f545c186145a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.205824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a80aa-17af-4303-b3f4-f545c186145a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.206129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a80aa-17af-4303-b3f4-f545c186145a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.206402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a80aa-17af-4303-b3f4-f545c186145a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.220697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a80aa-17af-4303-b3f4-f545c186145a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.222154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a80aa-17af-4303-b3f4-f545c186145a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-6f0a80aa-17af-4303-b3f4-f545c186145a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:43.237968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:43.239112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:43.240074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:43.240074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:43.240074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:43.240074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 283/568] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:43.240074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a80aa-17af-4303-b3f4-f545c186145a None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.273250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e6759-23b3-4b99-9c94-95bc622b74e6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.285476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:43.319702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa422f02-08c3-4bc9-ae0c-d5af58c8609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.320335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa422f02-08c3-4bc9-ae0c-d5af58c8609a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.320622 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa422f02-08c3-4bc9-ae0c-d5af58c8609a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.321654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa422f02-08c3-4bc9-ae0c-d5af58c8609a None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.321654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa422f02-08c3-4bc9-ae0c-d5af58c8609a None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:43.321879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa422f02-08c3-4bc9-ae0c-d5af58c8609a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.323513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa422f02-08c3-4bc9-ae0c-d5af58c8609a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.333680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 286/569] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:43.334620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa422f02-08c3-4bc9-ae0c-d5af58c8609a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.350159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:43.353060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a80aa-17af-4303-b3f4-f545c186145a None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.387464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fe928-fe1c-42da-ba61-d72b58e5a599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.388330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fe928-fe1c-42da-ba61-d72b58e5a599 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.388952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fe928-fe1c-42da-ba61-d72b58e5a599 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.389540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fe928-fe1c-42da-ba61-d72b58e5a599 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.390198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fe928-fe1c-42da-ba61-d72b58e5a599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:43.390530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fe928-fe1c-42da-ba61-d72b58e5a599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.392158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fe928-fe1c-42da-ba61-d72b58e5a599 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.404177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 284/570] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:43.405236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fe928-fe1c-42da-ba61-d72b58e5a599 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.461040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa422f02-08c3-4bc9-ae0c-d5af58c8609a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.497299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbd1e7-747b-430e-ab96-8816967081bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.498150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbd1e7-747b-430e-ab96-8816967081bf None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.498283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbd1e7-747b-430e-ab96-8816967081bf None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.498636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbd1e7-747b-430e-ab96-8816967081bf None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.499770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbd1e7-747b-430e-ab96-8816967081bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:43.499770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbd1e7-747b-430e-ab96-8816967081bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.502041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbbd1e7-747b-430e-ab96-8816967081bf None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.521233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 287/571] 10.222.0.82 () {56 vars in 1133 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:43.522117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbd1e7-747b-430e-ab96-8816967081bf None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.531812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fe928-fe1c-42da-ba61-d72b58e5a599 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.552399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:43.568535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc769998-4004-4da0-b15d-91625ad47c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.569255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc769998-4004-4da0-b15d-91625ad47c3a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.569446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc769998-4004-4da0-b15d-91625ad47c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.569720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc769998-4004-4da0-b15d-91625ad47c3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.580263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc769998-4004-4da0-b15d-91625ad47c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-bc769998-4004-4da0-b15d-91625ad47c3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:43.585505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:43.586532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 285/572] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:43.587427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc769998-4004-4da0-b15d-91625ad47c3a None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.612764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:43.624464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbd1e7-747b-430e-ab96-8816967081bf None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.659021 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bafd3-912f-4fcd-8ee9-bb3282ee249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.659619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bafd3-912f-4fcd-8ee9-bb3282ee249d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.659905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bafd3-912f-4fcd-8ee9-bb3282ee249d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.660177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bafd3-912f-4fcd-8ee9-bb3282ee249d None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.660817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bafd3-912f-4fcd-8ee9-bb3282ee249d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:43.661114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bafd3-912f-4fcd-8ee9-bb3282ee249d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.662523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bafd3-912f-4fcd-8ee9-bb3282ee249d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.674426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 288/573] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:43.675257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bafd3-912f-4fcd-8ee9-bb3282ee249d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.698654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc769998-4004-4da0-b15d-91625ad47c3a None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.736226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdfa88-4de8-489e-a93d-048e96b732cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.736872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdfa88-4de8-489e-a93d-048e96b732cc None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.737210 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdfa88-4de8-489e-a93d-048e96b732cc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.737490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdfa88-4de8-489e-a93d-048e96b732cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.748271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdfa88-4de8-489e-a93d-048e96b732cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.749667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdfa88-4de8-489e-a93d-048e96b732cc None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-07cdfa88-4de8-489e-a93d-048e96b732cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:43.765192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:43.766679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:43.767633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:43.767633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:43.767633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:43.767633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 286/574] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:43.767876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdfa88-4de8-489e-a93d-048e96b732cc None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.786976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:43.796083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bafd3-912f-4fcd-8ee9-bb3282ee249d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.835172 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8313c3-ad4a-4125-9fa4-9ba9cf0dfa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.835809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8313c3-ad4a-4125-9fa4-9ba9cf0dfa8f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.836094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8313c3-ad4a-4125-9fa4-9ba9cf0dfa8f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.836370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8313c3-ad4a-4125-9fa4-9ba9cf0dfa8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.843396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8313c3-ad4a-4125-9fa4-9ba9cf0dfa8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.845267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8313c3-ad4a-4125-9fa4-9ba9cf0dfa8f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.848334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-1e8313c3-ad4a-4125-9fa4-9ba9cf0dfa8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:43.856419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:43.863661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 289/575] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:43.864785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8313c3-ad4a-4125-9fa4-9ba9cf0dfa8f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.882468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdfa88-4de8-489e-a93d-048e96b732cc None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:43.922742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b451497-d544-445f-8fa6-9c4697176b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:43.924226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b451497-d544-445f-8fa6-9c4697176b61 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:43.924226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b451497-d544-445f-8fa6-9c4697176b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:43.924226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b451497-d544-445f-8fa6-9c4697176b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:43.935497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b451497-d544-445f-8fa6-9c4697176b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:43.936988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b451497-d544-445f-8fa6-9c4697176b61 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-4b451497-d544-445f-8fa6-9c4697176b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:43.956205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:43.957241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:43.958673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:43.958673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:43.958673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:43.959111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 287/576] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:43.959945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b451497-d544-445f-8fa6-9c4697176b61 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:43.979720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8313c3-ad4a-4125-9fa4-9ba9cf0dfa8f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.020892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c178da-ecf7-4308-9782-da126dd64084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.021520 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c178da-ecf7-4308-9782-da126dd64084 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.021798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c178da-ecf7-4308-9782-da126dd64084 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.022070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c178da-ecf7-4308-9782-da126dd64084 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.022865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c178da-ecf7-4308-9782-da126dd64084 None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:44.023070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c178da-ecf7-4308-9782-da126dd64084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.024589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c178da-ecf7-4308-9782-da126dd64084 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.034461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 290/577] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:44.035396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c178da-ecf7-4308-9782-da126dd64084 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.051477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.084096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b451497-d544-445f-8fa6-9c4697176b61 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.114484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.125525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dece0a6-67c3-43c2-96cd-86ca87f01a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.126611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dece0a6-67c3-43c2-96cd-86ca87f01a00 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.127031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dece0a6-67c3-43c2-96cd-86ca87f01a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.127507 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dece0a6-67c3-43c2-96cd-86ca87f01a00 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.137742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dece0a6-67c3-43c2-96cd-86ca87f01a00 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.139268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dece0a6-67c3-43c2-96cd-86ca87f01a00 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.151888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c178da-ecf7-4308-9782-da126dd64084 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-0dece0a6-67c3-43c2-96cd-86ca87f01a00 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:44.154539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:44.155796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:44.157787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 288/578] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:44.159599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dece0a6-67c3-43c2-96cd-86ca87f01a00 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.202703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82f3de-fdec-4092-bb5d-17b173ebb7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.203362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82f3de-fdec-4092-bb5d-17b173ebb7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.203639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82f3de-fdec-4092-bb5d-17b173ebb7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.203912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82f3de-fdec-4092-bb5d-17b173ebb7d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.204576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82f3de-fdec-4092-bb5d-17b173ebb7d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:44.204888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82f3de-fdec-4092-bb5d-17b173ebb7d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.206272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82f3de-fdec-4092-bb5d-17b173ebb7d8 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.221201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 291/579] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:44.222277 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82f3de-fdec-4092-bb5d-17b173ebb7d8 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.276089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dece0a6-67c3-43c2-96cd-86ca87f01a00 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.306625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.313257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d56fca-6cdd-41d7-bfda-6196f8791740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.313871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d56fca-6cdd-41d7-bfda-6196f8791740 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.314158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d56fca-6cdd-41d7-bfda-6196f8791740 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.314436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d56fca-6cdd-41d7-bfda-6196f8791740 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.315329 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d56fca-6cdd-41d7-bfda-6196f8791740 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:44.315637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d56fca-6cdd-41d7-bfda-6196f8791740 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.317097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d56fca-6cdd-41d7-bfda-6196f8791740 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.330400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 289/580] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:44.331341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d56fca-6cdd-41d7-bfda-6196f8791740 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.346002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82f3de-fdec-4092-bb5d-17b173ebb7d8 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.383114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.385388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92624ebd-d5b0-4261-8a05-7a1ecded428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.386006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92624ebd-d5b0-4261-8a05-7a1ecded428c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.386292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92624ebd-d5b0-4261-8a05-7a1ecded428c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.386601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92624ebd-d5b0-4261-8a05-7a1ecded428c None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.387252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92624ebd-d5b0-4261-8a05-7a1ecded428c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:44.387562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92624ebd-d5b0-4261-8a05-7a1ecded428c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.388959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92624ebd-d5b0-4261-8a05-7a1ecded428c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.402654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 292/581] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:44.403403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92624ebd-d5b0-4261-8a05-7a1ecded428c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.446487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d56fca-6cdd-41d7-bfda-6196f8791740 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.484344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8b90e-0694-49ec-a5f0-69cdbce8c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.485298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8b90e-0694-49ec-a5f0-69cdbce8c49f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.485298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8b90e-0694-49ec-a5f0-69cdbce8c49f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.485673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8b90e-0694-49ec-a5f0-69cdbce8c49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.496211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8b90e-0694-49ec-a5f0-69cdbce8c49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.497685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8b90e-0694-49ec-a5f0-69cdbce8c49f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-10b8b90e-0694-49ec-a5f0-69cdbce8c49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:44.522315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:44.528114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:44.529470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:44.529470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:44.529470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:44.529470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 290/582] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:44.529470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8b90e-0694-49ec-a5f0-69cdbce8c49f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.533031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92624ebd-d5b0-4261-8a05-7a1ecded428c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.571470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.576450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba87950-a91d-451c-87cc-7dd657d11cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.577067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba87950-a91d-451c-87cc-7dd657d11cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.577345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba87950-a91d-451c-87cc-7dd657d11cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.577619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba87950-a91d-451c-87cc-7dd657d11cb6 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.578898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba87950-a91d-451c-87cc-7dd657d11cb6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:44.583838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba87950-a91d-451c-87cc-7dd657d11cb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.583838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba87950-a91d-451c-87cc-7dd657d11cb6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.600488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 293/583] 10.222.0.82 () {56 vars in 1137 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:44.606446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba87950-a91d-451c-87cc-7dd657d11cb6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.638973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.646230 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8b90e-0694-49ec-a5f0-69cdbce8c49f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.687681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dd6e4-cd67-41b4-9974-2f36023eeb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.688314 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dd6e4-cd67-41b4-9974-2f36023eeb27 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.688592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dd6e4-cd67-41b4-9974-2f36023eeb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.688863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dd6e4-cd67-41b4-9974-2f36023eeb27 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.696470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571dd6e4-cd67-41b4-9974-2f36023eeb27 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.697931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571dd6e4-cd67-41b4-9974-2f36023eeb27 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-571dd6e4-cd67-41b4-9974-2f36023eeb27 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:44.710531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:44.712283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 291/584] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:44.714064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dd6e4-cd67-41b4-9974-2f36023eeb27 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.744274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba87950-a91d-451c-87cc-7dd657d11cb6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.784879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dafb3f-1877-4459-beb0-057c9eb9375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.785434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dafb3f-1877-4459-beb0-057c9eb9375f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.785808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dafb3f-1877-4459-beb0-057c9eb9375f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.786110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dafb3f-1877-4459-beb0-057c9eb9375f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.786979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dafb3f-1877-4459-beb0-057c9eb9375f None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=ad67f6ace29a4591bf2b8b4f119b3323) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:44.787259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dafb3f-1877-4459-beb0-057c9eb9375f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.788785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dafb3f-1877-4459-beb0-057c9eb9375f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.794791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 294/585] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/role_assignments?user.id=ad67f6ace29a4591bf2b8b4f119b3323&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:44.795625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dafb3f-1877-4459-beb0-057c9eb9375f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.845807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dd6e4-cd67-41b4-9974-2f36023eeb27 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.887661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:44.902683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e404a7-dde8-4c55-8633-82884200ec7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:44.903344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e404a7-dde8-4c55-8633-82884200ec7f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:44.903630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e404a7-dde8-4c55-8633-82884200ec7f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:44.904314 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e404a7-dde8-4c55-8633-82884200ec7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:44.914232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e404a7-dde8-4c55-8633-82884200ec7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:44.915843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e404a7-dde8-4c55-8633-82884200ec7f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-79e404a7-dde8-4c55-8633-82884200ec7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:44.929139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:44.930169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:44.931126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:44.931126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:44.931126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:44.931126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:44.931126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:44.931126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:44.931126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:44.931126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:44.935500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 292/586] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:44.938848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e404a7-dde8-4c55-8633-82884200ec7f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:44.957210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dafb3f-1877-4459-beb0-057c9eb9375f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:44.963123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:45.001548 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab31aa-bdca-4314-9d03-f87a003bcc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.002169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab31aa-bdca-4314-9d03-f87a003bcc21 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.002449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab31aa-bdca-4314-9d03-f87a003bcc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.002775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab31aa-bdca-4314-9d03-f87a003bcc21 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.003494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab31aa-bdca-4314-9d03-f87a003bcc21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:45.005556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab31aa-bdca-4314-9d03-f87a003bcc21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.007013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab31aa-bdca-4314-9d03-f87a003bcc21 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.022396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 295/587] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:45.023321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab31aa-bdca-4314-9d03-f87a003bcc21 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.087074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e404a7-dde8-4c55-8633-82884200ec7f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.139327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c785d2f-6b61-4447-9a2b-c164e357330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.140003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c785d2f-6b61-4447-9a2b-c164e357330d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.140249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c785d2f-6b61-4447-9a2b-c164e357330d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.140527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c785d2f-6b61-4447-9a2b-c164e357330d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.161997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c785d2f-6b61-4447-9a2b-c164e357330d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.163382 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c785d2f-6b61-4447-9a2b-c164e357330d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.165119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab31aa-bdca-4314-9d03-f87a003bcc21 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.184679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-0c785d2f-6b61-4447-9a2b-c164e357330d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:45.191126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:45.192169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:45.193141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:45.193141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:45.193141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:45.197443 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 293/588] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:45.204031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c785d2f-6b61-4447-9a2b-c164e357330d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.225431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7cb5-ad33-421d-a261-4d3c3348d320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.226403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7cb5-ad33-421d-a261-4d3c3348d320 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.226845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7cb5-ad33-421d-a261-4d3c3348d320 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.227165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7cb5-ad33-421d-a261-4d3c3348d320 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.228744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d7cb5-ad33-421d-a261-4d3c3348d320 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:45.229061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d7cb5-ad33-421d-a261-4d3c3348d320 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.231766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d7cb5-ad33-421d-a261-4d3c3348d320 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.277050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 296/589] 10.222.0.82 () {56 vars in 1137 bytes} [Wed Aug 24 03:30:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:45.277959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7cb5-ad33-421d-a261-4d3c3348d320 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.335065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:45.383569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c785d2f-6b61-4447-9a2b-c164e357330d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.449142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7cb5-ad33-421d-a261-4d3c3348d320 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.451451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e916c-4cee-4e86-a42c-1a519f120fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.452526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e916c-4cee-4e86-a42c-1a519f120fad None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.452949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e916c-4cee-4e86-a42c-1a519f120fad None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.453351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e916c-4cee-4e86-a42c-1a519f120fad None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.454433 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e916c-4cee-4e86-a42c-1a519f120fad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:45.454965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e916c-4cee-4e86-a42c-1a519f120fad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.457159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e916c-4cee-4e86-a42c-1a519f120fad None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.482270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 294/590] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:45.483283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e916c-4cee-4e86-a42c-1a519f120fad None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.504822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1369-c048-43f7-9cef-3685189eb2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.505570 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1369-c048-43f7-9cef-3685189eb2ae None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.505926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1369-c048-43f7-9cef-3685189eb2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.506277 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1369-c048-43f7-9cef-3685189eb2ae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.515668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede1369-c048-43f7-9cef-3685189eb2ae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.517645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede1369-c048-43f7-9cef-3685189eb2ae None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-4ede1369-c048-43f7-9cef-3685189eb2ae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:45.535958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:45.537515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 297/591] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:45.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1369-c048-43f7-9cef-3685189eb2ae None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.581723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:45.672105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e916c-4cee-4e86-a42c-1a519f120fad None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.691156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1369-c048-43f7-9cef-3685189eb2ae None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.709238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:45.718257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555d445-16a7-4e49-9bd2-2c8f098cabbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.719229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555d445-16a7-4e49-9bd2-2c8f098cabbc None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.719870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555d445-16a7-4e49-9bd2-2c8f098cabbc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.720059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555d445-16a7-4e49-9bd2-2c8f098cabbc None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.730764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555d445-16a7-4e49-9bd2-2c8f098cabbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:45.730764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555d445-16a7-4e49-9bd2-2c8f098cabbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.730764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555d445-16a7-4e49-9bd2-2c8f098cabbc None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.742395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 295/592] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:45.743161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1602a-8ee4-4fa3-a56e-415794eeb13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.743557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555d445-16a7-4e49-9bd2-2c8f098cabbc None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.743920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1602a-8ee4-4fa3-a56e-415794eeb13d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.744292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1602a-8ee4-4fa3-a56e-415794eeb13d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.744784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1602a-8ee4-4fa3-a56e-415794eeb13d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.764001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1602a-8ee4-4fa3-a56e-415794eeb13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.765764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1602a-8ee4-4fa3-a56e-415794eeb13d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-ecf1602a-8ee4-4fa3-a56e-415794eeb13d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:45.783672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:45.785049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:45.786056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:45.786056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:45.786056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:45.791558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 298/593] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:45.792821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1602a-8ee4-4fa3-a56e-415794eeb13d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:45.904582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555d445-16a7-4e49-9bd2-2c8f098cabbc None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.933502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:45.961198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1602a-8ee4-4fa3-a56e-415794eeb13d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:45.966131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064763a-46dd-4c9b-a0e5-c0c0572bb156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:45.967208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064763a-46dd-4c9b-a0e5-c0c0572bb156 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:45.967739 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064763a-46dd-4c9b-a0e5-c0c0572bb156 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:45.968205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064763a-46dd-4c9b-a0e5-c0c0572bb156 None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/0f34fe3fe912498b90dbf950e7fc999d/roles/9d16c3fb52ac40068ddddbf66411c519` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:45.987826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064763a-46dd-4c9b-a0e5-c0c0572bb156 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=0f34fe3fe912498b90dbf950e7fc999d, role_id=9d16c3fb52ac40068ddddbf66411c519)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:45.989843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064763a-46dd-4c9b-a0e5-c0c0572bb156 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.020506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3a4a2-14cc-493c-ac40-6592a699eefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.021169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3a4a2-14cc-493c-ac40-6592a699eefd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.021456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3a4a2-14cc-493c-ac40-6592a699eefd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.021735 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3a4a2-14cc-493c-ac40-6592a699eefd None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.022489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3a4a2-14cc-493c-ac40-6592a699eefd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:46.022844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3a4a2-14cc-493c-ac40-6592a699eefd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.024989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3a4a2-14cc-493c-ac40-6592a699eefd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.050883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 299/594] 10.222.0.82 () {56 vars in 1139 bytes} [Wed Aug 24 03:30:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:46.050883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3a4a2-14cc-493c-ac40-6592a699eefd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.075350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 296/595] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:30:45 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/0f34fe3fe912498b90dbf950e7fc999d/roles/9d16c3fb52ac40068ddddbf66411c519 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:46.084068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064763a-46dd-4c9b-a0e5-c0c0572bb156 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.147356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:46.254789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064763a-46dd-4c9b-a0e5-c0c0572bb156 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:46.272919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3a4a2-14cc-493c-ac40-6592a699eefd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:46.335114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5672a-de08-40c9-9fc3-d6ccdfb9d73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.336124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5672a-de08-40c9-9fc3-d6ccdfb9d73b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.336453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca1277-0891-4f2e-a65f-f4112e34bdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.336714 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5672a-de08-40c9-9fc3-d6ccdfb9d73b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.337121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5672a-de08-40c9-9fc3-d6ccdfb9d73b None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.337295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca1277-0891-4f2e-a65f-f4112e34bdfd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.337590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca1277-0891-4f2e-a65f-f4112e34bdfd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.337866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca1277-0891-4f2e-a65f-f4112e34bdfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.338415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5672a-de08-40c9-9fc3-d6ccdfb9d73b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:46.338952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5672a-de08-40c9-9fc3-d6ccdfb9d73b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.340859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5672a-de08-40c9-9fc3-d6ccdfb9d73b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.348653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca1277-0891-4f2e-a65f-f4112e34bdfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.349652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca1277-0891-4f2e-a65f-f4112e34bdfd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.359933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 297/596] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:46.360808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5672a-de08-40c9-9fc3-d6ccdfb9d73b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-15ca1277-0891-4f2e-a65f-f4112e34bdfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:46.365807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:46.366806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:46.367707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:46.367707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:46.367707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:46.367707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:46.367707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:46.367707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:46.367707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:46.367707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:46.369920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 300/597] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:46.370987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca1277-0891-4f2e-a65f-f4112e34bdfd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.375738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:46.532976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca1277-0891-4f2e-a65f-f4112e34bdfd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:46.540628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:46.546219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5672a-de08-40c9-9fc3-d6ccdfb9d73b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:46.594103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f85c6-8b0d-43b2-a197-778f7c6b10e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.594797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f85c6-8b0d-43b2-a197-778f7c6b10e1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.595079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f85c6-8b0d-43b2-a197-778f7c6b10e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.595352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f85c6-8b0d-43b2-a197-778f7c6b10e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.616004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81beaf0c-c888-423f-b9eb-8f464bc4b7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.616642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81beaf0c-c888-423f-b9eb-8f464bc4b7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.616928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81beaf0c-c888-423f-b9eb-8f464bc4b7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.617198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81beaf0c-c888-423f-b9eb-8f464bc4b7b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.618925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f85c6-8b0d-43b2-a197-778f7c6b10e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.620557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f85c6-8b0d-43b2-a197-778f7c6b10e1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.631301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81beaf0c-c888-423f-b9eb-8f464bc4b7b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.632790 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81beaf0c-c888-423f-b9eb-8f464bc4b7b4 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-ec0f85c6-8b0d-43b2-a197-778f7c6b10e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:46.648494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:46.649497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:46.650409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:46.650409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:46.650409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:46.656047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 298/598] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:46.656047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f85c6-8b0d-43b2-a197-778f7c6b10e1 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-81beaf0c-c888-423f-b9eb-8f464bc4b7b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:46.661409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:46.662336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:46.663285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:46.663285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:46.663285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:46.663285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:46.663285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:46.663285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:46.663285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:46.663285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:46.665422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 301/599] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:46.666294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81beaf0c-c888-423f-b9eb-8f464bc4b7b4 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.759107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:46.815682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81beaf0c-c888-423f-b9eb-8f464bc4b7b4 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:46.852017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f85c6-8b0d-43b2-a197-778f7c6b10e1 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:46.860449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9654f5-18a0-4e68-9ddf-7ea3d274555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.861328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9654f5-18a0-4e68-9ddf-7ea3d274555a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.861683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9654f5-18a0-4e68-9ddf-7ea3d274555a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.861915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9654f5-18a0-4e68-9ddf-7ea3d274555a None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.862759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9654f5-18a0-4e68-9ddf-7ea3d274555a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:46.862910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9654f5-18a0-4e68-9ddf-7ea3d274555a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.864261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9654f5-18a0-4e68-9ddf-7ea3d274555a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.882115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 302/600] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:46.882115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9654f5-18a0-4e68-9ddf-7ea3d274555a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.895554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553e52-731a-49f9-ab89-610c2428bc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:46.896574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553e52-731a-49f9-ab89-610c2428bc8b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.896854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553e52-731a-49f9-ab89-610c2428bc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.897393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553e52-731a-49f9-ab89-610c2428bc8b None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.898113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553e52-731a-49f9-ab89-610c2428bc8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:46.898421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553e52-731a-49f9-ab89-610c2428bc8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:46.900387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553e52-731a-49f9-ab89-610c2428bc8b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:46.925794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 299/601] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:46.925794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368572f-61de-4512-80b7-1699e2e3f382 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:46.925794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368572f-61de-4512-80b7-1699e2e3f382 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:46.925794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368572f-61de-4512-80b7-1699e2e3f382 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:46.928501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 300/602] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:46.929434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368572f-61de-4512-80b7-1699e2e3f382 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:46.943751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:47.014550 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9654f5-18a0-4e68-9ddf-7ea3d274555a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:47.045645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:47.065348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5b61a-43f4-44e0-bdd9-67f0dac2f192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:47.067227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5b61a-43f4-44e0-bdd9-67f0dac2f192 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:47.067488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5b61a-43f4-44e0-bdd9-67f0dac2f192 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:47.067785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5b61a-43f4-44e0-bdd9-67f0dac2f192 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:47.068471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5b61a-43f4-44e0-bdd9-67f0dac2f192 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:47.068835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5b61a-43f4-44e0-bdd9-67f0dac2f192 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:47.068930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368572f-61de-4512-80b7-1699e2e3f382 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:47.071303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5b61a-43f4-44e0-bdd9-67f0dac2f192 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:47.103441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 303/603] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:47.104850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5b61a-43f4-44e0-bdd9-67f0dac2f192 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:47.119500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0292b-d24e-42a1-91bc-e323e1c09d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:47.120180 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0292b-d24e-42a1-91bc-e323e1c09d4d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:47.120491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0292b-d24e-42a1-91bc-e323e1c09d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:47.120769 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0292b-d24e-42a1-91bc-e323e1c09d4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:47.130016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0292b-d24e-42a1-91bc-e323e1c09d4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:47.131732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0292b-d24e-42a1-91bc-e323e1c09d4d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-42c0292b-d24e-42a1-91bc-e323e1c09d4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:47.147530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:47.148688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:47.149809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:47.149809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:47.149809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:47.149809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:47.149809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:47.149809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:47.149809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:47.149809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:47.153627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 301/604] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:47.155117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece3e6a-4be1-40f4-852e-a50cdcdd6008 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:47.155410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece3e6a-4be1-40f4-852e-a50cdcdd6008 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:47.155715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece3e6a-4be1-40f4-852e-a50cdcdd6008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:47.178788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5ece3e6a-4be1-40f4-852e-a50cdcdd6008 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:47.180635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5ece3e6a-4be1-40f4-852e-a50cdcdd6008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:47.258368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5b61a-43f4-44e0-bdd9-67f0dac2f192 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:47.282981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:47.289025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 302/605] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:47.290346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece3e6a-4be1-40f4-852e-a50cdcdd6008 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:47.335554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06e9be-7f8f-4aef-9ac6-e09ae1fa7005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:47.336770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06e9be-7f8f-4aef-9ac6-e09ae1fa7005 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:47.337437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06e9be-7f8f-4aef-9ac6-e09ae1fa7005 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:47.338046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06e9be-7f8f-4aef-9ac6-e09ae1fa7005 None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/c89caed732274718aeaa0d8ffeef571b/roles/9d16c3fb52ac40068ddddbf66411c519` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:47.371757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06e9be-7f8f-4aef-9ac6-e09ae1fa7005 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=c89caed732274718aeaa0d8ffeef571b, role_id=9d16c3fb52ac40068ddddbf66411c519)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:47.373452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06e9be-7f8f-4aef-9ac6-e09ae1fa7005 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:47.432767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece3e6a-4be1-40f4-852e-a50cdcdd6008 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:47.454541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 304/606] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:30:47 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/c89caed732274718aeaa0d8ffeef571b/roles/9d16c3fb52ac40068ddddbf66411c519 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:47.454541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06e9be-7f8f-4aef-9ac6-e09ae1fa7005 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:47.530256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0edf4-8566-488f-a972-3b426e60238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:47.534760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0edf4-8566-488f-a972-3b426e60238f None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:47.536862 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0edf4-8566-488f-a972-3b426e60238f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:47.537548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0edf4-8566-488f-a972-3b426e60238f None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/4ff3f53ddbb04eb8916b9ccfdb03dd80/roles/9d16c3fb52ac40068ddddbf66411c519` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:47.560979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:47.593229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0edf4-8566-488f-a972-3b426e60238f None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=4ff3f53ddbb04eb8916b9ccfdb03dd80, role_id=9d16c3fb52ac40068ddddbf66411c519)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:47.598156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0edf4-8566-488f-a972-3b426e60238f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:47.676614 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06e9be-7f8f-4aef-9ac6-e09ae1fa7005 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:47.702468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 303/607] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:30:47 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/4ff3f53ddbb04eb8916b9ccfdb03dd80/roles/9d16c3fb52ac40068ddddbf66411c519 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:47.702468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0edf4-8566-488f-a972-3b426e60238f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:47.727256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df962d2b-53a7-4822-8cf7-898829adb0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:47.728348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df962d2b-53a7-4822-8cf7-898829adb0bb None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:47.728659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df962d2b-53a7-4822-8cf7-898829adb0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:47.729532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df962d2b-53a7-4822-8cf7-898829adb0bb None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:47.729771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df962d2b-53a7-4822-8cf7-898829adb0bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:47.730101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df962d2b-53a7-4822-8cf7-898829adb0bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:47.731714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df962d2b-53a7-4822-8cf7-898829adb0bb None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:47.747491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 305/608] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:47.757056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df962d2b-53a7-4822-8cf7-898829adb0bb None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:47.795005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:47.880639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:47.892165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0edf4-8566-488f-a972-3b426e60238f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:47.898538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df962d2b-53a7-4822-8cf7-898829adb0bb None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:47.982551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b77a4-361f-48df-aea4-302b59f42fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:47.983835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b77a4-361f-48df-aea4-302b59f42fcd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:47.984118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b77a4-361f-48df-aea4-302b59f42fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:47.984722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b77a4-361f-48df-aea4-302b59f42fcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:47.987261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12dcec-aa10-42da-b6ca-5245e8e7d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:47.987980 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12dcec-aa10-42da-b6ca-5245e8e7d266 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:47.988269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12dcec-aa10-42da-b6ca-5245e8e7d266 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:47.991051 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12dcec-aa10-42da-b6ca-5245e8e7d266 None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/f81cc77e85f84d358471dd9efab45cd4/roles/9d16c3fb52ac40068ddddbf66411c519` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.025107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12dcec-aa10-42da-b6ca-5245e8e7d266 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=f81cc77e85f84d358471dd9efab45cd4, role_id=9d16c3fb52ac40068ddddbf66411c519)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.026888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12dcec-aa10-42da-b6ca-5245e8e7d266 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.029833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b77a4-361f-48df-aea4-302b59f42fcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-d15b77a4-361f-48df-aea4-302b59f42fcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:48.035675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:48.036635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:48.037504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:48.046624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 304/609] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:48.049586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b77a4-361f-48df-aea4-302b59f42fcd None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.135965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 306/610] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:30:47 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/f81cc77e85f84d358471dd9efab45cd4/roles/9d16c3fb52ac40068ddddbf66411c519 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:30:48.208567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b77a4-361f-48df-aea4-302b59f42fcd None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.229487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:48.326484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07d94b-2203-4d46-abfb-87de1df0f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.327416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07d94b-2203-4d46-abfb-87de1df0f37e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.327810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07d94b-2203-4d46-abfb-87de1df0f37e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.328145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07d94b-2203-4d46-abfb-87de1df0f37e None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.329144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07d94b-2203-4d46-abfb-87de1df0f37e None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:48.329458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07d94b-2203-4d46-abfb-87de1df0f37e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.330888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07d94b-2203-4d46-abfb-87de1df0f37e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.350171 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 305/611] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:48.356704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12dcec-aa10-42da-b6ca-5245e8e7d266 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.382727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd96354-d919-49e0-86c7-dca555b0dafc None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.383365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd96354-d919-49e0-86c7-dca555b0dafc None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.383847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd96354-d919-49e0-86c7-dca555b0dafc None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.385772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 306/612] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:48.400189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b48480-4e28-410d-b1b4-0d15777a028d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.400810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b48480-4e28-410d-b1b4-0d15777a028d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.401301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b48480-4e28-410d-b1b4-0d15777a028d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.427905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-56b48480-4e28-410d-b1b4-0d15777a028d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:48.429603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-56b48480-4e28-410d-b1b4-0d15777a028d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:48.529722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12dcec-aa10-42da-b6ca-5245e8e7d266 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.590344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01475479-168d-43cd-9575-02e200a6ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.592201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01475479-168d-43cd-9575-02e200a6ec10 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.593579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01475479-168d-43cd-9575-02e200a6ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.594959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01475479-168d-43cd-9575-02e200a6ec10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.608815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 307/613] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:48.614488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01475479-168d-43cd-9575-02e200a6ec10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.616259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01475479-168d-43cd-9575-02e200a6ec10 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.618285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-01475479-168d-43cd-9575-02e200a6ec10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:48.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:48.634300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:48.635355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:48.635355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:48.635355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:48.637364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 307/614] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:48.643203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b48480-4e28-410d-b1b4-0d15777a028d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.852794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b48480-4e28-410d-b1b4-0d15777a028d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:48.854915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:48.916208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e8895-ef6f-4216-9935-5d24edae4dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:48.917000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e8895-ef6f-4216-9935-5d24edae4dcb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:48.917276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e8895-ef6f-4216-9935-5d24edae4dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:48.917555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e8895-ef6f-4216-9935-5d24edae4dcb None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:48.918220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e8895-ef6f-4216-9935-5d24edae4dcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:48.918621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e8895-ef6f-4216-9935-5d24edae4dcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:48.919967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e8895-ef6f-4216-9935-5d24edae4dcb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:48.940268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 308/615] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:48.946313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01475479-168d-43cd-9575-02e200a6ec10 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:48.966338 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e8895-ef6f-4216-9935-5d24edae4dcb None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.182100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e8895-ef6f-4216-9935-5d24edae4dcb None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.185774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01475479-168d-43cd-9575-02e200a6ec10 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.263032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f0950-a68b-4194-8eba-2878ebc0212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.263691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f0950-a68b-4194-8eba-2878ebc0212e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.263970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f0950-a68b-4194-8eba-2878ebc0212e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.264243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f0950-a68b-4194-8eba-2878ebc0212e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.294595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f0950-a68b-4194-8eba-2878ebc0212e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-773f0950-a68b-4194-8eba-2878ebc0212e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:49.301409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:49.302482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:49.303437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:49.304093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe8c3-3d36-4157-8606-98481ff26e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.304093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe8c3-3d36-4157-8606-98481ff26e08 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.304254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe8c3-3d36-4157-8606-98481ff26e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.304964 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe8c3-3d36-4157-8606-98481ff26e08 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.318545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fe8c3-3d36-4157-8606-98481ff26e08 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.318912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 309/616] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:49.319781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f0950-a68b-4194-8eba-2878ebc0212e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.321636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fe8c3-3d36-4157-8606-98481ff26e08 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-9a5fe8c3-3d36-4157-8606-98481ff26e08 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:49.345917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:49.347041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:30:49.348145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:49.358364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 308/617] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:30:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:49.365773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe8c3-3d36-4157-8606-98481ff26e08 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.382558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:49.494081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:49.570960 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f0950-a68b-4194-8eba-2878ebc0212e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.606182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe8c3-3d36-4157-8606-98481ff26e08 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.652036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b2cc-ab62-4f19-bd1b-ffb2bca0d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.653689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b2cc-ab62-4f19-bd1b-ffb2bca0d96d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.654721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b2cc-ab62-4f19-bd1b-ffb2bca0d96d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.654721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b2cc-ab62-4f19-bd1b-ffb2bca0d96d None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.654992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236b2cc-ab62-4f19-bd1b-ffb2bca0d96d None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:49.655253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236b2cc-ab62-4f19-bd1b-ffb2bca0d96d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.658533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236b2cc-ab62-4f19-bd1b-ffb2bca0d96d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.687791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 310/618] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:49.690537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b2cc-ab62-4f19-bd1b-ffb2bca0d96d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.690672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62125de1-0dbb-4b05-ac70-de45da4fad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.690672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62125de1-0dbb-4b05-ac70-de45da4fad86 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.690672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62125de1-0dbb-4b05-ac70-de45da4fad86 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.690999 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62125de1-0dbb-4b05-ac70-de45da4fad86 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.691620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62125de1-0dbb-4b05-ac70-de45da4fad86 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:49.694529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62125de1-0dbb-4b05-ac70-de45da4fad86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.694529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62125de1-0dbb-4b05-ac70-de45da4fad86 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.733164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 309/619] 10.222.0.82 () {56 vars in 1135 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:49.739792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62125de1-0dbb-4b05-ac70-de45da4fad86 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:49.884842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:49.903391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b2cc-ab62-4f19-bd1b-ffb2bca0d96d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:49.952185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5f3f8-8aa0-4407-9070-5a9c43df7fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:49.952844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5f3f8-8aa0-4407-9070-5a9c43df7fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:49.953146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5f3f8-8aa0-4407-9070-5a9c43df7fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:49.953405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5f3f8-8aa0-4407-9070-5a9c43df7fc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:49.968163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5f3f8-8aa0-4407-9070-5a9c43df7fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:49.974183 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5f3f8-8aa0-4407-9070-5a9c43df7fc9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-bef5f3f8-8aa0-4407-9070-5a9c43df7fc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:49.996592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:49.997624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:49.999022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:49.999022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:49.999022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:49.999022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 311/620] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:50.005469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5f3f8-8aa0-4407-9070-5a9c43df7fc9 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.028193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62125de1-0dbb-4b05-ac70-de45da4fad86 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.062167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.072708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e330a6-0f27-4cef-b6d2-80888bd8a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.073948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e330a6-0f27-4cef-b6d2-80888bd8a474 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.074605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e330a6-0f27-4cef-b6d2-80888bd8a474 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.074882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e330a6-0f27-4cef-b6d2-80888bd8a474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.089109 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e330a6-0f27-4cef-b6d2-80888bd8a474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.091629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e330a6-0f27-4cef-b6d2-80888bd8a474 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-84e330a6-0f27-4cef-b6d2-80888bd8a474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:50.111859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:50.112876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:50.114285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:50.114285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:50.114285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:50.114285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 310/621] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:50.126535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e330a6-0f27-4cef-b6d2-80888bd8a474 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.276430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e330a6-0f27-4cef-b6d2-80888bd8a474 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.334796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5f3f8-8aa0-4407-9070-5a9c43df7fc9 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.339693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fe457-9fa4-44e8-9b12-a3391cdaef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.340612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fe457-9fa4-44e8-9b12-a3391cdaef3a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.341105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fe457-9fa4-44e8-9b12-a3391cdaef3a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.341529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fe457-9fa4-44e8-9b12-a3391cdaef3a None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.342410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fe457-9fa4-44e8-9b12-a3391cdaef3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:50.342896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fe457-9fa4-44e8-9b12-a3391cdaef3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.345390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fe457-9fa4-44e8-9b12-a3391cdaef3a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.363792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 311/622] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:30:50.365430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db9738-f3a6-4be2-991d-d8e7f8b6c496 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.366107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db9738-f3a6-4be2-991d-d8e7f8b6c496 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.366556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db9738-f3a6-4be2-991d-d8e7f8b6c496 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.368552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 312/623] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:50.374855 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039a85e-e5ce-430e-b8b3-261043ea391c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.376130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039a85e-e5ce-430e-b8b3-261043ea391c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.377428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039a85e-e5ce-430e-b8b3-261043ea391c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.379435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 313/624] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:50.380467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039a85e-e5ce-430e-b8b3-261043ea391c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.392436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4f57-1c48-4c6b-a04e-476242fa7ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.393063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4f57-1c48-4c6b-a04e-476242fa7ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.393345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4f57-1c48-4c6b-a04e-476242fa7ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.393621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4f57-1c48-4c6b-a04e-476242fa7ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.394281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4f57-1c48-4c6b-a04e-476242fa7ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:50.394699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4f57-1c48-4c6b-a04e-476242fa7ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.396373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4f57-1c48-4c6b-a04e-476242fa7ca3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.399416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.413398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 312/625] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:50.414267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.415647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bf9b3-a9e0-4dbd-9e07-4761d28041c4 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.416004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bf9b3-a9e0-4dbd-9e07-4761d28041c4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.416207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bf9b3-a9e0-4dbd-9e07-4761d28041c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.440588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-812bf9b3-a9e0-4dbd-9e07-4761d28041c4 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:50.440588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-812bf9b3-a9e0-4dbd-9e07-4761d28041c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.532474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039a85e-e5ce-430e-b8b3-261043ea391c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.539543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 313/626] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:50.541446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c314fad-241a-48c6-ad70-86fbcdfcbb24 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.541907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c314fad-241a-48c6-ad70-86fbcdfcbb24 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.542383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c314fad-241a-48c6-ad70-86fbcdfcbb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.571506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-0c314fad-241a-48c6-ad70-86fbcdfcbb24 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:50.572756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0c314fad-241a-48c6-ad70-86fbcdfcbb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.599289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279f620-1bd6-4d6b-899c-1dbfa702771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.601227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279f620-1bd6-4d6b-899c-1dbfa702771d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.601771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279f620-1bd6-4d6b-899c-1dbfa702771d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.602235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279f620-1bd6-4d6b-899c-1dbfa702771d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.617239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279f620-1bd6-4d6b-899c-1dbfa702771d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.621273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279f620-1bd6-4d6b-899c-1dbfa702771d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-f279f620-1bd6-4d6b-899c-1dbfa702771d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:50.644589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:50.645580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 314/627] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:50.647205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279f620-1bd6-4d6b-899c-1dbfa702771d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.667352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 314/628] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:50.668731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c3d88-2822-4874-8ec1-770bb0dad1f6 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.669030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c3d88-2822-4874-8ec1-770bb0dad1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.669318 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c3d88-2822-4874-8ec1-770bb0dad1f6 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.676808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 315/629] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:50.676808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c3d88-2822-4874-8ec1-770bb0dad1f6 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:50.749192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.811450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c3d88-2822-4874-8ec1-770bb0dad1f6 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.815596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.862491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279f620-1bd6-4d6b-899c-1dbfa702771d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:50.868300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c00523-40cc-4320-9cac-8fda7ca39006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.868956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c00523-40cc-4320-9cac-8fda7ca39006 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.869237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c00523-40cc-4320-9cac-8fda7ca39006 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.869508 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c00523-40cc-4320-9cac-8fda7ca39006 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.870238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c00523-40cc-4320-9cac-8fda7ca39006 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:50.870587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c00523-40cc-4320-9cac-8fda7ca39006 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.872140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c00523-40cc-4320-9cac-8fda7ca39006 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.887651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 316/630] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:50.888946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98563b-135e-4f8e-b36b-fc2e3e7c14f6 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.889228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98563b-135e-4f8e-b36b-fc2e3e7c14f6 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.889502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98563b-135e-4f8e-b36b-fc2e3e7c14f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.909392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5e98563b-135e-4f8e-b36b-fc2e3e7c14f6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:50.910662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5e98563b-135e-4f8e-b36b-fc2e3e7c14f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.926033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db2e50-c7d1-4068-a297-74dde434eebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:50.931830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db2e50-c7d1-4068-a297-74dde434eebf None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.931830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db2e50-c7d1-4068-a297-74dde434eebf None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.931830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db2e50-c7d1-4068-a297-74dde434eebf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.938915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db2e50-c7d1-4068-a297-74dde434eebf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:50.941607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db2e50-c7d1-4068-a297-74dde434eebf None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-80db2e50-c7d1-4068-a297-74dde434eebf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:50.957208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:50.958612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:30:50.959778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:50.963469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 315/631] 10.222.0.82 () {56 vars in 1093 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:50.964721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db2e50-c7d1-4068-a297-74dde434eebf None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.034366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 317/632] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:51.038169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98563b-135e-4f8e-b36b-fc2e3e7c14f6 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.090064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.135610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db2e50-c7d1-4068-a297-74dde434eebf None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.157932 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98563b-135e-4f8e-b36b-fc2e3e7c14f6 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.174418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4521284-4074-4e71-b6e7-cb8d49c5dd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.175079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4521284-4074-4e71-b6e7-cb8d49c5dd22 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.175361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4521284-4074-4e71-b6e7-cb8d49c5dd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.175641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4521284-4074-4e71-b6e7-cb8d49c5dd22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.189663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4521284-4074-4e71-b6e7-cb8d49c5dd22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-a4521284-4074-4e71-b6e7-cb8d49c5dd22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:51.197655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:51.198729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:51.199628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:51.200336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c16074-641e-497f-8033-1062595e7305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.201250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16074-641e-497f-8033-1062595e7305 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.201355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16074-641e-497f-8033-1062595e7305 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.201618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16074-641e-497f-8033-1062595e7305 None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/ad67f6ace29a4591bf2b8b4f119b3323/roles/9d16c3fb52ac40068ddddbf66411c519` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.204897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 316/633] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:51.206334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4521284-4074-4e71-b6e7-cb8d49c5dd22 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.213169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.221059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c16074-641e-497f-8033-1062595e7305 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=ad67f6ace29a4591bf2b8b4f119b3323, role_id=9d16c3fb52ac40068ddddbf66411c519)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.222850 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c16074-641e-497f-8033-1062595e7305 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.319229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 318/634] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:30:51 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/ad67f6ace29a4591bf2b8b4f119b3323/roles/9d16c3fb52ac40068ddddbf66411c519 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:30:51.320892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c16074-641e-497f-8033-1062595e7305 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.334419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4521284-4074-4e71-b6e7-cb8d49c5dd22 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.381771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240ebee-04d7-4888-b813-3badcb65232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.383244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240ebee-04d7-4888-b813-3badcb65232f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.383534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240ebee-04d7-4888-b813-3badcb65232f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.383809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240ebee-04d7-4888-b813-3badcb65232f None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.384578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240ebee-04d7-4888-b813-3badcb65232f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:51.384888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240ebee-04d7-4888-b813-3badcb65232f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.386706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240ebee-04d7-4888-b813-3badcb65232f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.415543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 317/635] 10.222.0.82 () {56 vars in 1133 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:51.416531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240ebee-04d7-4888-b813-3badcb65232f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.471839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.477938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c16074-641e-497f-8033-1062595e7305 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.504179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.525712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417ef72-ef8e-47fd-9228-b2d9e0080d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.526360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417ef72-ef8e-47fd-9228-b2d9e0080d73 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.526709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417ef72-ef8e-47fd-9228-b2d9e0080d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.526996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417ef72-ef8e-47fd-9228-b2d9e0080d73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.545399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417ef72-ef8e-47fd-9228-b2d9e0080d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-7417ef72-ef8e-47fd-9228-b2d9e0080d73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:51.553217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:51.554586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 319/636] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:51.555713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417ef72-ef8e-47fd-9228-b2d9e0080d73 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.558319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240ebee-04d7-4888-b813-3badcb65232f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.605371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fef257-5b89-4a4b-9f3c-d65a02f6ded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.606231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fef257-5b89-4a4b-9f3c-d65a02f6ded9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.606864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fef257-5b89-4a4b-9f3c-d65a02f6ded9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.607143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fef257-5b89-4a4b-9f3c-d65a02f6ded9 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.607822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fef257-5b89-4a4b-9f3c-d65a02f6ded9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:51.608129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fef257-5b89-4a4b-9f3c-d65a02f6ded9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.609635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fef257-5b89-4a4b-9f3c-d65a02f6ded9 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.620125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 318/637] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:51.621057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fef257-5b89-4a4b-9f3c-d65a02f6ded9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.703997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417ef72-ef8e-47fd-9228-b2d9e0080d73 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.759747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2638028-7ec1-4294-8e04-9d5fc3f5e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.760532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2638028-7ec1-4294-8e04-9d5fc3f5e5db None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.760825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2638028-7ec1-4294-8e04-9d5fc3f5e5db None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.761105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2638028-7ec1-4294-8e04-9d5fc3f5e5db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.762782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fef257-5b89-4a4b-9f3c-d65a02f6ded9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.779946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2638028-7ec1-4294-8e04-9d5fc3f5e5db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-b2638028-7ec1-4294-8e04-9d5fc3f5e5db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:51.785997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:51.787047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:51.787946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:51.790668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 320/638] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:51.791580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2638028-7ec1-4294-8e04-9d5fc3f5e5db None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.799930 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.817562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.820175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb37ce9-3a2f-4ddc-bed5-1a4ef53fa4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.820806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb37ce9-3a2f-4ddc-bed5-1a4ef53fa4ce None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.821087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb37ce9-3a2f-4ddc-bed5-1a4ef53fa4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.821360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb37ce9-3a2f-4ddc-bed5-1a4ef53fa4ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.923966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2638028-7ec1-4294-8e04-9d5fc3f5e5db None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.925637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb37ce9-3a2f-4ddc-bed5-1a4ef53fa4ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.928227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb37ce9-3a2f-4ddc-bed5-1a4ef53fa4ce None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.970087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15ee06-b2d9-4be8-9a51-656490dae14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.970087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15ee06-b2d9-4be8-9a51-656490dae14a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.970087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15ee06-b2d9-4be8-9a51-656490dae14a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.970648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15ee06-b2d9-4be8-9a51-656490dae14a None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.971431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15ee06-b2d9-4be8-9a51-656490dae14a None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:51.971900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15ee06-b2d9-4be8-9a51-656490dae14a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.973434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15ee06-b2d9-4be8-9a51-656490dae14a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.990606 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 321/639] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:51.992129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15ee06-b2d9-4be8-9a51-656490dae14a None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.086519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 319/640] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:52.087967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb37ce9-3a2f-4ddc-bed5-1a4ef53fa4ce None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.109196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.123942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.139034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15ee06-b2d9-4be8-9a51-656490dae14a None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.193219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bcf26-5c9e-47a1-a2d7-00377b742cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.194244 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bcf26-5c9e-47a1-a2d7-00377b742cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.194644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bcf26-5c9e-47a1-a2d7-00377b742cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.195024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bcf26-5c9e-47a1-a2d7-00377b742cc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.210303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bcf26-5c9e-47a1-a2d7-00377b742cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.211900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bcf26-5c9e-47a1-a2d7-00377b742cc5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-d50bcf26-5c9e-47a1-a2d7-00377b742cc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:52.229953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:52.230978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:52.232532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:52.232532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:52.232532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:52.232532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 322/641] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:52.232532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bcf26-5c9e-47a1-a2d7-00377b742cc5 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.244099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb37ce9-3a2f-4ddc-bed5-1a4ef53fa4ce None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.296842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f47cdc-7bd4-4a3e-bbff-960ac394593f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.298349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f47cdc-7bd4-4a3e-bbff-960ac394593f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.298703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f47cdc-7bd4-4a3e-bbff-960ac394593f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.298982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f47cdc-7bd4-4a3e-bbff-960ac394593f None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.299982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f47cdc-7bd4-4a3e-bbff-960ac394593f None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:52.300605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f47cdc-7bd4-4a3e-bbff-960ac394593f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.302009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f47cdc-7bd4-4a3e-bbff-960ac394593f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.314898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 320/642] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:52.315813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f47cdc-7bd4-4a3e-bbff-960ac394593f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.370941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bcf26-5c9e-47a1-a2d7-00377b742cc5 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.410436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.417267 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513387c9-1c5e-4105-97c8-72b8fb7430de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.417910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513387c9-1c5e-4105-97c8-72b8fb7430de None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.418227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513387c9-1c5e-4105-97c8-72b8fb7430de None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.418916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513387c9-1c5e-4105-97c8-72b8fb7430de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.423982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.432041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513387c9-1c5e-4105-97c8-72b8fb7430de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.433703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513387c9-1c5e-4105-97c8-72b8fb7430de None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.446278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f47cdc-7bd4-4a3e-bbff-960ac394593f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-513387c9-1c5e-4105-97c8-72b8fb7430de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:52.455678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:52.456763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:52.457684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:52.457684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:52.457684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:52.462053 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 323/643] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:52.463695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513387c9-1c5e-4105-97c8-72b8fb7430de None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.500681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41efa9-35c8-43e3-863c-b69621cba267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.501540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41efa9-35c8-43e3-863c-b69621cba267 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.501822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41efa9-35c8-43e3-863c-b69621cba267 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.502093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41efa9-35c8-43e3-863c-b69621cba267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.513371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41efa9-35c8-43e3-863c-b69621cba267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.514893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41efa9-35c8-43e3-863c-b69621cba267 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-fd41efa9-35c8-43e3-863c-b69621cba267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:52.530737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:52.532372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:52.534086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:52.534086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:52.534086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:52.534086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 321/644] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:52.534086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41efa9-35c8-43e3-863c-b69621cba267 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.596345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513387c9-1c5e-4105-97c8-72b8fb7430de None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.644695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ea757-bba6-490b-91a6-a81ce6eec3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.645757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ea757-bba6-490b-91a6-a81ce6eec3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.646245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ea757-bba6-490b-91a6-a81ce6eec3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.646755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ea757-bba6-490b-91a6-a81ce6eec3e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.647583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ea757-bba6-490b-91a6-a81ce6eec3e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:52.648360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ea757-bba6-490b-91a6-a81ce6eec3e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.649755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ea757-bba6-490b-91a6-a81ce6eec3e9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.668767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 324/645] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:52.668767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ea757-bba6-490b-91a6-a81ce6eec3e9 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.682916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41efa9-35c8-43e3-863c-b69621cba267 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.714648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.726877 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fa07b-b694-4f4f-8573-c190d98de772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.727515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fa07b-b694-4f4f-8573-c190d98de772 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.729975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fa07b-b694-4f4f-8573-c190d98de772 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.729975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fa07b-b694-4f4f-8573-c190d98de772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.736944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:52.741749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fa07b-b694-4f4f-8573-c190d98de772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.743424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fa07b-b694-4f4f-8573-c190d98de772 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-658fa07b-b694-4f4f-8573-c190d98de772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:52.761263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:52.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:52.763686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:52.763686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:52.763686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:52.763686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 322/646] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:52.764287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fa07b-b694-4f4f-8573-c190d98de772 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.807516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ea757-bba6-490b-91a6-a81ce6eec3e9 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.853357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fb291-9dc7-48c9-8408-e69de3b2d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.854334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fb291-9dc7-48c9-8408-e69de3b2d2bd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.854888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fb291-9dc7-48c9-8408-e69de3b2d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.855364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fb291-9dc7-48c9-8408-e69de3b2d2bd None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.856247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fb291-9dc7-48c9-8408-e69de3b2d2bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:52.856763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fb291-9dc7-48c9-8408-e69de3b2d2bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.858417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fb291-9dc7-48c9-8408-e69de3b2d2bd None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.875680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 325/647] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:52.875680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fb291-9dc7-48c9-8408-e69de3b2d2bd None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:52.891485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fa07b-b694-4f4f-8573-c190d98de772 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:52.935746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2725d1-faf5-4fda-81d0-d554579dda17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:52.936376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2725d1-faf5-4fda-81d0-d554579dda17 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:52.936661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2725d1-faf5-4fda-81d0-d554579dda17 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:52.936937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2725d1-faf5-4fda-81d0-d554579dda17 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:52.939787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2725d1-faf5-4fda-81d0-d554579dda17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:52.939787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2725d1-faf5-4fda-81d0-d554579dda17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:52.941681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2725d1-faf5-4fda-81d0-d554579dda17 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:52.962302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 323/648] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:52.962302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2725d1-faf5-4fda-81d0-d554579dda17 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.010639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fb291-9dc7-48c9-8408-e69de3b2d2bd None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.012644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.044460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.055712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e6fd0-c112-42a9-b9c4-035ccedba9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.056457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e6fd0-c112-42a9-b9c4-035ccedba9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.057925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e6fd0-c112-42a9-b9c4-035ccedba9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.057925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e6fd0-c112-42a9-b9c4-035ccedba9d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.069525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e6fd0-c112-42a9-b9c4-035ccedba9d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.071097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e6fd0-c112-42a9-b9c4-035ccedba9d4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-455e6fd0-c112-42a9-b9c4-035ccedba9d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:53.085735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:53.086953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 326/649] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:53.092772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e6fd0-c112-42a9-b9c4-035ccedba9d4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.120078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2725d1-faf5-4fda-81d0-d554579dda17 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.176350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeddcab-c451-439e-84f3-8cbddd681a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.177303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeddcab-c451-439e-84f3-8cbddd681a68 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.178356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeddcab-c451-439e-84f3-8cbddd681a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.178668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeddcab-c451-439e-84f3-8cbddd681a68 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.179329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeddcab-c451-439e-84f3-8cbddd681a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:53.179638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeddcab-c451-439e-84f3-8cbddd681a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.181090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeddcab-c451-439e-84f3-8cbddd681a68 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.196869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 324/650] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:53.197723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeddcab-c451-439e-84f3-8cbddd681a68 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.247967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e6fd0-c112-42a9-b9c4-035ccedba9d4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.295657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc510584-8f1c-449a-a143-08a7d02ef2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.296217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc510584-8f1c-449a-a143-08a7d02ef2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.296558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc510584-8f1c-449a-a143-08a7d02ef2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.296833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc510584-8f1c-449a-a143-08a7d02ef2e5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.306048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc510584-8f1c-449a-a143-08a7d02ef2e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.307966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc510584-8f1c-449a-a143-08a7d02ef2e5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-fc510584-8f1c-449a-a143-08a7d02ef2e5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:53.321044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:53.322044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:53.323046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 327/651] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:53.324154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc510584-8f1c-449a-a143-08a7d02ef2e5 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.340168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeddcab-c451-439e-84f3-8cbddd681a68 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.356810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.364720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.382334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fce7aa-2769-43db-9877-c820ddaaafc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.383101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fce7aa-2769-43db-9877-c820ddaaafc1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.383453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fce7aa-2769-43db-9877-c820ddaaafc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.384014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fce7aa-2769-43db-9877-c820ddaaafc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.394261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fce7aa-2769-43db-9877-c820ddaaafc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.395850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fce7aa-2769-43db-9877-c820ddaaafc1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-d1fce7aa-2769-43db-9877-c820ddaaafc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:53.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:53.413675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 325/652] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:53.414765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fce7aa-2769-43db-9877-c820ddaaafc1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.452687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc510584-8f1c-449a-a143-08a7d02ef2e5 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.493018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681b829-1418-49ae-ac42-49af42c27ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.493643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b829-1418-49ae-ac42-49af42c27ddf None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.493929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b829-1418-49ae-ac42-49af42c27ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.494207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b829-1418-49ae-ac42-49af42c27ddf None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.495110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681b829-1418-49ae-ac42-49af42c27ddf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:53.495423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681b829-1418-49ae-ac42-49af42c27ddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.496879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681b829-1418-49ae-ac42-49af42c27ddf None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.516374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 328/653] 10.222.0.82 () {56 vars in 1137 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:53.517032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681b829-1418-49ae-ac42-49af42c27ddf None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.533227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fce7aa-2769-43db-9877-c820ddaaafc1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.570866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be2f90-27a8-49a8-9baa-47fb3e3d1b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.571437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be2f90-27a8-49a8-9baa-47fb3e3d1b04 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.571722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be2f90-27a8-49a8-9baa-47fb3e3d1b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.572017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be2f90-27a8-49a8-9baa-47fb3e3d1b04 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.581815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be2f90-27a8-49a8-9baa-47fb3e3d1b04 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.583463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be2f90-27a8-49a8-9baa-47fb3e3d1b04 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-c2be2f90-27a8-49a8-9baa-47fb3e3d1b04 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:53.595339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:53.596405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:53.597391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 326/654] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:53.598005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f3d1-48c1-442b-9f64-48c64ce21ed1 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.598005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f3d1-48c1-442b-9f64-48c64ce21ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.598700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f3d1-48c1-442b-9f64-48c64ce21ed1 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.604381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 327/655] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:53.605245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0f3d1-48c1-442b-9f64-48c64ce21ed1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.616080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.636984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681b829-1418-49ae-ac42-49af42c27ddf None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.642336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.674742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064c737-7f59-4a43-b75a-ac6b9db0dee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.675364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064c737-7f59-4a43-b75a-ac6b9db0dee8 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.675651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064c737-7f59-4a43-b75a-ac6b9db0dee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.675929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064c737-7f59-4a43-b75a-ac6b9db0dee8 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.676668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064c737-7f59-4a43-b75a-ac6b9db0dee8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:53.676983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064c737-7f59-4a43-b75a-ac6b9db0dee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.678639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064c737-7f59-4a43-b75a-ac6b9db0dee8 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.699151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 329/656] 10.222.0.82 () {56 vars in 1137 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:53.700072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064c737-7f59-4a43-b75a-ac6b9db0dee8 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.757080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0f3d1-48c1-442b-9f64-48c64ce21ed1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.800536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67040108-66fb-49e5-a7d5-81d818ffd682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.801433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040108-66fb-49e5-a7d5-81d818ffd682 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.801715 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040108-66fb-49e5-a7d5-81d818ffd682 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.801987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040108-66fb-49e5-a7d5-81d818ffd682 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.802966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67040108-66fb-49e5-a7d5-81d818ffd682 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:53.803280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67040108-66fb-49e5-a7d5-81d818ffd682 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.804732 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67040108-66fb-49e5-a7d5-81d818ffd682 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.818495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 328/657] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:53.819307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67040108-66fb-49e5-a7d5-81d818ffd682 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:53.825424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064c737-7f59-4a43-b75a-ac6b9db0dee8 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.866109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c641f-7c9a-478d-95c4-205a2a7803f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.866782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c641f-7c9a-478d-95c4-205a2a7803f6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.867064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c641f-7c9a-478d-95c4-205a2a7803f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.867354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c641f-7c9a-478d-95c4-205a2a7803f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.876955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.927016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:53.937195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67040108-66fb-49e5-a7d5-81d818ffd682 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:53.949112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c641f-7c9a-478d-95c4-205a2a7803f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.950744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c641f-7c9a-478d-95c4-205a2a7803f6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.973616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f3b6a-ab1f-4fdf-b7d4-05e5057aabe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:53.980095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f3b6a-ab1f-4fdf-b7d4-05e5057aabe8 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.980095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f3b6a-ab1f-4fdf-b7d4-05e5057aabe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.980095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f3b6a-ab1f-4fdf-b7d4-05e5057aabe8 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.980095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f3b6a-ab1f-4fdf-b7d4-05e5057aabe8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:53.980095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f3b6a-ab1f-4fdf-b7d4-05e5057aabe8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:53.980095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f3b6a-ab1f-4fdf-b7d4-05e5057aabe8 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:53.997280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 329/658] 10.222.0.82 () {56 vars in 1139 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:53.998412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fbe2c-2df3-4bd6-98f0-9313ae04113c None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.998731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fbe2c-2df3-4bd6-98f0-9313ae04113c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.999005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fbe2c-2df3-4bd6-98f0-9313ae04113c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.021991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-331fbe2c-2df3-4bd6-98f0-9313ae04113c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:54.023673 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-331fbe2c-2df3-4bd6-98f0-9313ae04113c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.077783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 330/659] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:54.079620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c641f-7c9a-478d-95c4-205a2a7803f6 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.108148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 330/660] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:54.109270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fbe2c-2df3-4bd6-98f0-9313ae04113c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.127744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.208907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.215880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c641f-7c9a-478d-95c4-205a2a7803f6 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.241173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fbe2c-2df3-4bd6-98f0-9313ae04113c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.263289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af8a4f-568a-4e92-a161-f194664c6d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.263917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af8a4f-568a-4e92-a161-f194664c6d16 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.264206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af8a4f-568a-4e92-a161-f194664c6d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.264488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af8a4f-568a-4e92-a161-f194664c6d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.288719 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4020ca-d0a7-4bcd-af8d-4939fc69297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.289704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4020ca-d0a7-4bcd-af8d-4939fc69297f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.290189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4020ca-d0a7-4bcd-af8d-4939fc69297f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.290671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4020ca-d0a7-4bcd-af8d-4939fc69297f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.291659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4020ca-d0a7-4bcd-af8d-4939fc69297f None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=0f34fe3fe912498b90dbf950e7fc999d) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:54.292121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4020ca-d0a7-4bcd-af8d-4939fc69297f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.293929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4020ca-d0a7-4bcd-af8d-4939fc69297f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.301896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 331/661] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/role_assignments?user.id=0f34fe3fe912498b90dbf950e7fc999d&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:54.302783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4020ca-d0a7-4bcd-af8d-4939fc69297f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.378023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af8a4f-568a-4e92-a161-f194664c6d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.379487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af8a4f-568a-4e92-a161-f194664c6d16 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.434819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.436518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4020ca-d0a7-4bcd-af8d-4939fc69297f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.486138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0e342-e3bc-4093-9292-2420f04bf718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.487995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0e342-e3bc-4093-9292-2420f04bf718 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.487995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0e342-e3bc-4093-9292-2420f04bf718 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.487995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0e342-e3bc-4093-9292-2420f04bf718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.505881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.522629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 331/662] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:54.523586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af8a4f-568a-4e92-a161-f194664c6d16 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.597844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0e342-e3bc-4093-9292-2420f04bf718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.600062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0e342-e3bc-4093-9292-2420f04bf718 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.663221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af8a4f-568a-4e92-a161-f194664c6d16 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.713752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fcbe3-2eba-4ea6-98f5-cb09a1cd6e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.714965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fcbe3-2eba-4ea6-98f5-cb09a1cd6e24 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.715398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fcbe3-2eba-4ea6-98f5-cb09a1cd6e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.715820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fcbe3-2eba-4ea6-98f5-cb09a1cd6e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.726817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.727334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 332/663] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:54.728995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0e342-e3bc-4093-9292-2420f04bf718 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.729302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118fcbe3-2eba-4ea6-98f5-cb09a1cd6e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.731160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118fcbe3-2eba-4ea6-98f5-cb09a1cd6e24 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-118fcbe3-2eba-4ea6-98f5-cb09a1cd6e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:54.749315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:54.751804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:54.752801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:54.752801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:54.752801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:54.758650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 332/664] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:54.759876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fcbe3-2eba-4ea6-98f5-cb09a1cd6e24 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.780548 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.851164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0e342-e3bc-4093-9292-2420f04bf718 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.893388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27782c51-0dbc-470e-9d01-c7791553676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.894039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27782c51-0dbc-470e-9d01-c7791553676b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.894252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fcbe3-2eba-4ea6-98f5-cb09a1cd6e24 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.894362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27782c51-0dbc-470e-9d01-c7791553676b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.894659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27782c51-0dbc-470e-9d01-c7791553676b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.909110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27782c51-0dbc-470e-9d01-c7791553676b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-27782c51-0dbc-470e-9d01-c7791553676b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:54.914253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:54.915464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:30:54.916337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:54.920311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 333/665] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:54.921216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27782c51-0dbc-470e-9d01-c7791553676b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.935726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50fddf-9e01-4e66-b1ec-6100b8658c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.936438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50fddf-9e01-4e66-b1ec-6100b8658c28 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.936803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50fddf-9e01-4e66-b1ec-6100b8658c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.937080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50fddf-9e01-4e66-b1ec-6100b8658c28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.950373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50fddf-9e01-4e66-b1ec-6100b8658c28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.952307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50fddf-9e01-4e66-b1ec-6100b8658c28 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-4d50fddf-9e01-4e66-b1ec-6100b8658c28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:54.976150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:54.977463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:54.978412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:54.978412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:54.978412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:54.978412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 333/666] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:54.979425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50fddf-9e01-4e66-b1ec-6100b8658c28 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.040153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:55.066423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27782c51-0dbc-470e-9d01-c7791553676b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.068642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:55.113683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50fddf-9e01-4e66-b1ec-6100b8658c28 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.117742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fd808-a089-4afb-94eb-f2eb089aee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.118397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fd808-a089-4afb-94eb-f2eb089aee6b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.118738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fd808-a089-4afb-94eb-f2eb089aee6b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.119104 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fd808-a089-4afb-94eb-f2eb089aee6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.133115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fd808-a089-4afb-94eb-f2eb089aee6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.135154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fd808-a089-4afb-94eb-f2eb089aee6b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-b29fd808-a089-4afb-94eb-f2eb089aee6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:55.155592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:55.157188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:55.158093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:55.158093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:55.158093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:55.158093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 334/667] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:55.160283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fd808-a089-4afb-94eb-f2eb089aee6b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.163444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf5534-d7de-46bc-bc64-753058c55a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.164124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf5534-d7de-46bc-bc64-753058c55a5c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.164463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf5534-d7de-46bc-bc64-753058c55a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.168511 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf5534-d7de-46bc-bc64-753058c55a5c None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.169204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf5534-d7de-46bc-bc64-753058c55a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:55.169523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf5534-d7de-46bc-bc64-753058c55a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.170976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf5534-d7de-46bc-bc64-753058c55a5c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.190908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 334/668] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:55.192084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf5534-d7de-46bc-bc64-753058c55a5c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.296254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fd808-a089-4afb-94eb-f2eb089aee6b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.322397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf5534-d7de-46bc-bc64-753058c55a5c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.324067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:55.337114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87887bc2-862c-45b0-9f35-b559538fbb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.337731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87887bc2-862c-45b0-9f35-b559538fbb96 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.339097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87887bc2-862c-45b0-9f35-b559538fbb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.339097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87887bc2-862c-45b0-9f35-b559538fbb96 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.339097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87887bc2-862c-45b0-9f35-b559538fbb96 None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:55.339334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87887bc2-862c-45b0-9f35-b559538fbb96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.340600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87887bc2-862c-45b0-9f35-b559538fbb96 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.355325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 335/669] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:55.355325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87887bc2-862c-45b0-9f35-b559538fbb96 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.368215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505cb58-1230-4682-b540-4e7c937e0e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.369572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505cb58-1230-4682-b540-4e7c937e0e6e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.370039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505cb58-1230-4682-b540-4e7c937e0e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.370865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505cb58-1230-4682-b540-4e7c937e0e6e None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.372018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505cb58-1230-4682-b540-4e7c937e0e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:55.372337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505cb58-1230-4682-b540-4e7c937e0e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.373899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505cb58-1230-4682-b540-4e7c937e0e6e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.383816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:55.388939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 335/670] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:55.389790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505cb58-1230-4682-b540-4e7c937e0e6e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.490619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87887bc2-862c-45b0-9f35-b559538fbb96 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.519103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505cb58-1230-4682-b540-4e7c937e0e6e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.539062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e8873-abb1-4efe-993f-738133362291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.540012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e8873-abb1-4efe-993f-738133362291 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.540524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e8873-abb1-4efe-993f-738133362291 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.541156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e8873-abb1-4efe-993f-738133362291 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.542127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e8873-abb1-4efe-993f-738133362291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:55.542858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e8873-abb1-4efe-993f-738133362291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.544534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e8873-abb1-4efe-993f-738133362291 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.560065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0820c-623b-42e3-b403-7d62fb122272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.560722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0820c-623b-42e3-b403-7d62fb122272 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.561010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0820c-623b-42e3-b403-7d62fb122272 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.561289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0820c-623b-42e3-b403-7d62fb122272 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.563508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 336/671] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:55.565076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e8873-abb1-4efe-993f-738133362291 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.574905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0820c-623b-42e3-b403-7d62fb122272 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.576385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0820c-623b-42e3-b403-7d62fb122272 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-3af0820c-623b-42e3-b403-7d62fb122272 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:55.590024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:55.591024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:55.591928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:55.591928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:55.591928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:55.591928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:55.591928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:55.591928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:55.591928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:55.591928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:55.594527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 336/672] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:55.595413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0820c-623b-42e3-b403-7d62fb122272 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.629393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:55.667022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:55.691867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e8873-abb1-4efe-993f-738133362291 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.730080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0820c-623b-42e3-b403-7d62fb122272 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.737512 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c1b58-ab3c-467f-b1ac-7c61ece9abd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.738263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c1b58-ab3c-467f-b1ac-7c61ece9abd7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.738659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c1b58-ab3c-467f-b1ac-7c61ece9abd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.739004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c1b58-ab3c-467f-b1ac-7c61ece9abd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.755200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c1b58-ab3c-467f-b1ac-7c61ece9abd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.757124 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c1b58-ab3c-467f-b1ac-7c61ece9abd7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-506c1b58-ab3c-467f-b1ac-7c61ece9abd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:55.775133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:55.776199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:55.777155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:55.777155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:55.777155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:55.777711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ae150-81f7-4d4e-82c4-4de0034e7b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.778612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ae150-81f7-4d4e-82c4-4de0034e7b4c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.778741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 337/673] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:55.779199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ae150-81f7-4d4e-82c4-4de0034e7b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.779644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c1b58-ab3c-467f-b1ac-7c61ece9abd7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.779847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ae150-81f7-4d4e-82c4-4de0034e7b4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.788742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ae150-81f7-4d4e-82c4-4de0034e7b4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.790210 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ae150-81f7-4d4e-82c4-4de0034e7b4c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-700ae150-81f7-4d4e-82c4-4de0034e7b4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:55.806745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:55.807723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:55.808693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:55.808693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:55.808693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:55.808693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:55.808693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:55.808693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:55.808693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:55.808693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:55.810878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 337/674] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:55.812156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ae150-81f7-4d4e-82c4-4de0034e7b4c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:55.905969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c1b58-ab3c-467f-b1ac-7c61ece9abd7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.942961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:55.946578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:55.956140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431cab5-d4e4-40a0-a5eb-d5406b2d60a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:55.957068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431cab5-d4e4-40a0-a5eb-d5406b2d60a3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:55.957442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431cab5-d4e4-40a0-a5eb-d5406b2d60a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:55.958286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431cab5-d4e4-40a0-a5eb-d5406b2d60a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:55.965076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ae150-81f7-4d4e-82c4-4de0034e7b4c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:55.973385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431cab5-d4e4-40a0-a5eb-d5406b2d60a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:55.975194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431cab5-d4e4-40a0-a5eb-d5406b2d60a3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-4431cab5-d4e4-40a0-a5eb-d5406b2d60a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:55.989274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:55.990285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 338/675] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:55.991752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431cab5-d4e4-40a0-a5eb-d5406b2d60a3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.023321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b230466-1f75-4e55-bbc0-bf5c2d9e39f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.024045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b230466-1f75-4e55-bbc0-bf5c2d9e39f3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.024158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b230466-1f75-4e55-bbc0-bf5c2d9e39f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.024429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b230466-1f75-4e55-bbc0-bf5c2d9e39f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.025135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b230466-1f75-4e55-bbc0-bf5c2d9e39f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:56.025796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b230466-1f75-4e55-bbc0-bf5c2d9e39f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.026895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b230466-1f75-4e55-bbc0-bf5c2d9e39f3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.041144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 338/676] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:56.044421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b230466-1f75-4e55-bbc0-bf5c2d9e39f3 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.145769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431cab5-d4e4-40a0-a5eb-d5406b2d60a3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.167067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b230466-1f75-4e55-bbc0-bf5c2d9e39f3 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.189669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f39698-d8d9-41e6-81c7-b370ebaae4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.190286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f39698-d8d9-41e6-81c7-b370ebaae4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.190680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f39698-d8d9-41e6-81c7-b370ebaae4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.191016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f39698-d8d9-41e6-81c7-b370ebaae4b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.191674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f39698-d8d9-41e6-81c7-b370ebaae4b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:56.191979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f39698-d8d9-41e6-81c7-b370ebaae4b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.193618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f39698-d8d9-41e6-81c7-b370ebaae4b0 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.213795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 339/677] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:56.215416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f39698-d8d9-41e6-81c7-b370ebaae4b0 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.218559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca4444-ca77-41c2-a232-7425c0c5d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.219776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca4444-ca77-41c2-a232-7425c0c5d57f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.220068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca4444-ca77-41c2-a232-7425c0c5d57f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.220506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca4444-ca77-41c2-a232-7425c0c5d57f None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.221215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca4444-ca77-41c2-a232-7425c0c5d57f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:56.221528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca4444-ca77-41c2-a232-7425c0c5d57f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.223040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca4444-ca77-41c2-a232-7425c0c5d57f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.243741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 339/678] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:56.244709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca4444-ca77-41c2-a232-7425c0c5d57f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.249278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:56.281872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:56.340387 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f39698-d8d9-41e6-81c7-b370ebaae4b0 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.381959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feada0c8-eec1-4c65-93e0-0204280711d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.382609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feada0c8-eec1-4c65-93e0-0204280711d9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.382883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feada0c8-eec1-4c65-93e0-0204280711d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.383158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feada0c8-eec1-4c65-93e0-0204280711d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.383875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feada0c8-eec1-4c65-93e0-0204280711d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:56.384511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feada0c8-eec1-4c65-93e0-0204280711d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.386324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feada0c8-eec1-4c65-93e0-0204280711d9 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.389255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca4444-ca77-41c2-a232-7425c0c5d57f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.402276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 340/679] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:56.403470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feada0c8-eec1-4c65-93e0-0204280711d9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.444262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcb4e5-9848-4d2f-8061-26ebb8d568ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.444637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcb4e5-9848-4d2f-8061-26ebb8d568ad None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.444912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcb4e5-9848-4d2f-8061-26ebb8d568ad None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.445190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcb4e5-9848-4d2f-8061-26ebb8d568ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.445922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcb4e5-9848-4d2f-8061-26ebb8d568ad None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=c89caed732274718aeaa0d8ffeef571b) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:56.446233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcb4e5-9848-4d2f-8061-26ebb8d568ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.447979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcb4e5-9848-4d2f-8061-26ebb8d568ad None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.454999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 340/680] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:56 2022] GET /identity/v3/role_assignments?user.id=c89caed732274718aeaa0d8ffeef571b&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:56.455824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcb4e5-9848-4d2f-8061-26ebb8d568ad None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.530009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feada0c8-eec1-4c65-93e0-0204280711d9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.542186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:56.559395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:56.584914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0db6bb-12b7-4b88-92b1-742252eb1b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.587647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0db6bb-12b7-4b88-92b1-742252eb1b5c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.587880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0db6bb-12b7-4b88-92b1-742252eb1b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.588518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0db6bb-12b7-4b88-92b1-742252eb1b5c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.608176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0db6bb-12b7-4b88-92b1-742252eb1b5c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.610085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0db6bb-12b7-4b88-92b1-742252eb1b5c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-2a0db6bb-12b7-4b88-92b1-742252eb1b5c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:56.629693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:56.631353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:56.632389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 341/681] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:30:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:56.633645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab126f-07a5-41bf-89e4-2633a13168d6 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.634220 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab126f-07a5-41bf-89e4-2633a13168d6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.634499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab126f-07a5-41bf-89e4-2633a13168d6 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.635849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 342/682] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:56.638543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab126f-07a5-41bf-89e4-2633a13168d6 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.638670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcb4e5-9848-4d2f-8061-26ebb8d568ad None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.696069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a0bff-edc9-4647-bba0-d38afcc91047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.697041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a0bff-edc9-4647-bba0-d38afcc91047 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.697736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a0bff-edc9-4647-bba0-d38afcc91047 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.697736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a0bff-edc9-4647-bba0-d38afcc91047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.698442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a0bff-edc9-4647-bba0-d38afcc91047 None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=4ff3f53ddbb04eb8916b9ccfdb03dd80) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:56.699002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a0bff-edc9-4647-bba0-d38afcc91047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.700467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a0bff-edc9-4647-bba0-d38afcc91047 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.715095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 341/683] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:56 2022] GET /identity/v3/role_assignments?user.id=4ff3f53ddbb04eb8916b9ccfdb03dd80&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:56.716443 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a0bff-edc9-4647-bba0-d38afcc91047 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:56.833717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab126f-07a5-41bf-89e4-2633a13168d6 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:56.889669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a07b63-bff0-4b47-b4f2-6245af175119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:56.890300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a07b63-bff0-4b47-b4f2-6245af175119 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.890619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a07b63-bff0-4b47-b4f2-6245af175119 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.892964 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a07b63-bff0-4b47-b4f2-6245af175119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.893870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a07b63-bff0-4b47-b4f2-6245af175119 None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=f81cc77e85f84d358471dd9efab45cd4) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:56.894267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a07b63-bff0-4b47-b4f2-6245af175119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:56.896420 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a07b63-bff0-4b47-b4f2-6245af175119 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:56.906215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 343/684] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:56 2022] GET /identity/v3/role_assignments?user.id=f81cc77e85f84d358471dd9efab45cd4&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:56.908103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef70124-255f-4a0e-bf62-c23a45340396 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.908103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef70124-255f-4a0e-bf62-c23a45340396 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.908412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef70124-255f-4a0e-bf62-c23a45340396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.944299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-8ef70124-255f-4a0e-bf62-c23a45340396 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:56.945716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8ef70124-255f-4a0e-bf62-c23a45340396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:56.994150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a0bff-edc9-4647-bba0-d38afcc91047 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.011973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:57.092451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:57.107478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d990b-f2e6-4853-8987-57bdba1bca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.109222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d990b-f2e6-4853-8987-57bdba1bca9a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.109222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d990b-f2e6-4853-8987-57bdba1bca9a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.110019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d990b-f2e6-4853-8987-57bdba1bca9a None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.111247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d990b-f2e6-4853-8987-57bdba1bca9a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:57.112748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d990b-f2e6-4853-8987-57bdba1bca9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.115865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d990b-f2e6-4853-8987-57bdba1bca9a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.122020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 344/685] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:57.148585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 342/686] 10.222.0.82 () {56 vars in 1135 bytes} [Wed Aug 24 03:30:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:57.154863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef70124-255f-4a0e-bf62-c23a45340396 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.282242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef70124-255f-4a0e-bf62-c23a45340396 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.324541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4888b3-0711-4ba3-8ce1-df1e2b10829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.325596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4888b3-0711-4ba3-8ce1-df1e2b10829d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.326099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4888b3-0711-4ba3-8ce1-df1e2b10829d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.326601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4888b3-0711-4ba3-8ce1-df1e2b10829d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.366574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:57.406314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4888b3-0711-4ba3-8ce1-df1e2b10829d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.407819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4888b3-0711-4ba3-8ce1-df1e2b10829d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.514640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 345/687] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:30:57.517196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d990b-f2e6-4853-8987-57bdba1bca9a None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.614962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4888b3-0711-4ba3-8ce1-df1e2b10829d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.692294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d990b-f2e6-4853-8987-57bdba1bca9a None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.776730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:57.795067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc558d-a876-4d02-9596-88b792026bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.796105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc558d-a876-4d02-9596-88b792026bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.796637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc558d-a876-4d02-9596-88b792026bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.798209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc558d-a876-4d02-9596-88b792026bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.842538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc558d-a876-4d02-9596-88b792026bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.843425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc558d-a876-4d02-9596-88b792026bc6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-92bc558d-a876-4d02-9596-88b792026bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:57.896501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:57.897536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:57.899105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:57.899105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:57.899105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:57.906529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 343/688] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:57.906529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc558d-a876-4d02-9596-88b792026bc6 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.944613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4888b3-0711-4ba3-8ce1-df1e2b10829d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.988516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:58.106557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc558d-a876-4d02-9596-88b792026bc6 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.170529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753b00b-7447-4354-b6b4-139b466bf05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.170529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753b00b-7447-4354-b6b4-139b466bf05e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.170529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753b00b-7447-4354-b6b4-139b466bf05e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.170529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753b00b-7447-4354-b6b4-139b466bf05e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.239886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af33bd3-a997-45ea-bd22-eed2eb7b518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.240877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af33bd3-a997-45ea-bd22-eed2eb7b518c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.240877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af33bd3-a997-45ea-bd22-eed2eb7b518c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.241056 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af33bd3-a997-45ea-bd22-eed2eb7b518c None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.248516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af33bd3-a997-45ea-bd22-eed2eb7b518c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:58.248924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af33bd3-a997-45ea-bd22-eed2eb7b518c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.252397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af33bd3-a997-45ea-bd22-eed2eb7b518c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.258553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753b00b-7447-4354-b6b4-139b466bf05e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-2753b00b-7447-4354-b6b4-139b466bf05e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:30:58.260495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:58.261410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:30:58.262273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:58.274129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 346/689] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 657 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:58.286534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753b00b-7447-4354-b6b4-139b466bf05e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.302081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 344/690] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:58.311777 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af33bd3-a997-45ea-bd22-eed2eb7b518c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.550868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:58.554373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af33bd3-a997-45ea-bd22-eed2eb7b518c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.598588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b57a1-43b7-4d71-b863-5cda9ac7e741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.600733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b57a1-43b7-4d71-b863-5cda9ac7e741 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.601023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b57a1-43b7-4d71-b863-5cda9ac7e741 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.601297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b57a1-43b7-4d71-b863-5cda9ac7e741 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.620157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b57a1-43b7-4d71-b863-5cda9ac7e741 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.621655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b57a1-43b7-4d71-b863-5cda9ac7e741 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-9d8b57a1-43b7-4d71-b863-5cda9ac7e741 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:30:58.645457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:58.646448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:58.648038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:30:58.648038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:30:58.648038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:30:58.648038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:30:58.648038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:30:58.648038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:30:58.648038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:30:58.648038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:30:58.648038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 345/691] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:30:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:58.654694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753b00b-7447-4354-b6b4-139b466bf05e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.655456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b57a1-43b7-4d71-b863-5cda9ac7e741 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.680543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:58.697141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0bd23-7525-4e40-a54c-828f69b1c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.698089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0bd23-7525-4e40-a54c-828f69b1c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.698653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0bd23-7525-4e40-a54c-828f69b1c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.700311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0bd23-7525-4e40-a54c-828f69b1c4d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.702126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0bd23-7525-4e40-a54c-828f69b1c4d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:58.702676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0bd23-7525-4e40-a54c-828f69b1c4d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.706267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0bd23-7525-4e40-a54c-828f69b1c4d5 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.729817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 347/692] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:30:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:58.735954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0bd23-7525-4e40-a54c-828f69b1c4d5 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.811076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b57a1-43b7-4d71-b863-5cda9ac7e741 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.863314 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb975f-27ad-42bc-8906-73cb7db1ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.864579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb975f-27ad-42bc-8906-73cb7db1ae6a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.865509 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb975f-27ad-42bc-8906-73cb7db1ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.866270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb975f-27ad-42bc-8906-73cb7db1ae6a None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.867443 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb975f-27ad-42bc-8906-73cb7db1ae6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:58.867939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb975f-27ad-42bc-8906-73cb7db1ae6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.870176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb975f-27ad-42bc-8906-73cb7db1ae6a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:58.887577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 346/693] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:58.889708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c671d-729b-459f-8c27-c82ccc66b049 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.890264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c671d-729b-459f-8c27-c82ccc66b049 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.891320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c671d-729b-459f-8c27-c82ccc66b049 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.893998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 347/694] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:58.895288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c671d-729b-459f-8c27-c82ccc66b049 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:58.921023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0bd23-7525-4e40-a54c-828f69b1c4d5 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:58.930889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:58.971560 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccff919-a4d6-4951-8f6e-485ac1876244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:58.972329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccff919-a4d6-4951-8f6e-485ac1876244 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:58.972684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccff919-a4d6-4951-8f6e-485ac1876244 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:58.973053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccff919-a4d6-4951-8f6e-485ac1876244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:58.988890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccff919-a4d6-4951-8f6e-485ac1876244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:58.990474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccff919-a4d6-4951-8f6e-485ac1876244 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-7ccff919-a4d6-4951-8f6e-485ac1876244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:30:59.012541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:59.013569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:30:59.014525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:30:59.014525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:30:59.014525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:59.018593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 348/695] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:30:59.022397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3691b7-270c-4587-8a80-d4099bb84937 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.022712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3691b7-270c-4587-8a80-d4099bb84937 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.022990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3691b7-270c-4587-8a80-d4099bb84937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.048156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-ec3691b7-270c-4587-8a80-d4099bb84937 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:59.048528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c671d-729b-459f-8c27-c82ccc66b049 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.050222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ec3691b7-270c-4587-8a80-d4099bb84937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.096441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:59.098332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3af4ed-e04c-4ee1-a4b3-70d4345aed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.099041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3af4ed-e04c-4ee1-a4b3-70d4345aed90 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.099310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3af4ed-e04c-4ee1-a4b3-70d4345aed90 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.099586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3af4ed-e04c-4ee1-a4b3-70d4345aed90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.100848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3af4ed-e04c-4ee1-a4b3-70d4345aed90 None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=ad67f6ace29a4591bf2b8b4f119b3323) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:59.101163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3af4ed-e04c-4ee1-a4b3-70d4345aed90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.102672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3af4ed-e04c-4ee1-a4b3-70d4345aed90 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.112209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 348/696] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:30:58 2022] GET /identity/v3/role_assignments?user.id=ad67f6ace29a4591bf2b8b4f119b3323&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:59.113145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3af4ed-e04c-4ee1-a4b3-70d4345aed90 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.159052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 349/697] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:59.226062 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:59.245301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3af4ed-e04c-4ee1-a4b3-70d4345aed90 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.285277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864fac1-5e78-4310-a918-4e5ad844c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.285914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864fac1-5e78-4310-a918-4e5ad844c2fd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.286199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864fac1-5e78-4310-a918-4e5ad844c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.286475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864fac1-5e78-4310-a918-4e5ad844c2fd None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.287343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864fac1-5e78-4310-a918-4e5ad844c2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:59.287652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864fac1-5e78-4310-a918-4e5ad844c2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.289070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864fac1-5e78-4310-a918-4e5ad844c2fd None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.303366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 349/698] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:59.309006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3691b7-270c-4587-8a80-d4099bb84937 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.459800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3691b7-270c-4587-8a80-d4099bb84937 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.516867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734a093-06a1-4c0d-b389-f9a66e952ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.517757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a093-06a1-4c0d-b389-f9a66e952ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.518171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a093-06a1-4c0d-b389-f9a66e952ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.518687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a093-06a1-4c0d-b389-f9a66e952ac1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.530844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734a093-06a1-4c0d-b389-f9a66e952ac1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.533599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734a093-06a1-4c0d-b389-f9a66e952ac1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-2734a093-06a1-4c0d-b389-f9a66e952ac1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:30:59.548001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:59.549423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:30:59.550644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:59.553870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 350/699] 10.222.0.82 () {56 vars in 1093 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:59.560881 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864fac1-5e78-4310-a918-4e5ad844c2fd None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.576411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734a093-06a1-4c0d-b389-f9a66e952ac1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.620449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:59.711390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734a093-06a1-4c0d-b389-f9a66e952ac1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.720930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864fac1-5e78-4310-a918-4e5ad844c2fd None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.749340 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362586-6181-4d72-b0ac-bd8f99da7c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.749997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362586-6181-4d72-b0ac-bd8f99da7c6a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.750399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362586-6181-4d72-b0ac-bd8f99da7c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.750702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362586-6181-4d72-b0ac-bd8f99da7c6a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.751304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96362586-6181-4d72-b0ac-bd8f99da7c6a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.752695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96362586-6181-4d72-b0ac-bd8f99da7c6a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.762096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6a756-be45-482a-9097-5da9c62b2432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.762864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6a756-be45-482a-9097-5da9c62b2432 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.763251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6a756-be45-482a-9097-5da9c62b2432 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.763631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6a756-be45-482a-9097-5da9c62b2432 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-96362586-6181-4d72-b0ac-bd8f99da7c6a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:30:59.764065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:30:59.765273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:30:59.766380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 351/700] 10.222.0.82 () {56 vars in 1129 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:30:59.767370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17351-f44b-427d-aac8-f28b63538bdd None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.767370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17351-f44b-427d-aac8-f28b63538bdd None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.767884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17351-f44b-427d-aac8-f28b63538bdd None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.769441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 352/701] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:59.770203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6a756-be45-482a-9097-5da9c62b2432 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:59.770645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6a756-be45-482a-9097-5da9c62b2432 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.770749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc928a-031b-4343-bba8-2eb79e4d778f None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.770936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc928a-031b-4343-bba8-2eb79e4d778f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.771212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc928a-031b-4343-bba8-2eb79e4d778f None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.772197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6a756-be45-482a-9097-5da9c62b2432 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.782766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 353/702] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:30:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:59.782766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d777bb-fd79-4496-ae51-2482f7730d12 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.782766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d777bb-fd79-4496-ae51-2482f7730d12 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.782766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d777bb-fd79-4496-ae51-2482f7730d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.799192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 350/703] 10.222.0.82 () {56 vars in 1133 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:59.800231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6a756-be45-482a-9097-5da9c62b2432 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:59.804551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:59.807169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-34d777bb-fd79-4496-ae51-2482f7730d12 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:59.808405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-34d777bb-fd79-4496-ae51-2482f7730d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.896694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 354/704] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:30:59.898442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671ecac-e5a5-4f2a-a57b-b22298fe272e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.899034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671ecac-e5a5-4f2a-a57b-b22298fe272e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.899518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671ecac-e5a5-4f2a-a57b-b22298fe272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.922686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-a671ecac-e5a5-4f2a-a57b-b22298fe272e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:59.922686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a671ecac-e5a5-4f2a-a57b-b22298fe272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:59.933142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6a756-be45-482a-9097-5da9c62b2432 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:59.937928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:59.979968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988f86-91a6-41a6-9ca5-5d0304402738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:59.986472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988f86-91a6-41a6-9ca5-5d0304402738 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:59.986472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988f86-91a6-41a6-9ca5-5d0304402738 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:59.986472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988f86-91a6-41a6-9ca5-5d0304402738 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:59.986472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad988f86-91a6-41a6-9ca5-5d0304402738 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:30:59.986472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad988f86-91a6-41a6-9ca5-5d0304402738 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:59.986472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad988f86-91a6-41a6-9ca5-5d0304402738 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:59.991699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 351/705] 10.222.0.82 () {56 vars in 1124 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:30:59.992528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988f86-91a6-41a6-9ca5-5d0304402738 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.010651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 355/706] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:31:00.011550 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671ecac-e5a5-4f2a-a57b-b22298fe272e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.109662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.171588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988f86-91a6-41a6-9ca5-5d0304402738 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.199496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671ecac-e5a5-4f2a-a57b-b22298fe272e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.220803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa802c4b-af5f-4d9a-9398-975693f1fc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.221746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa802c4b-af5f-4d9a-9398-975693f1fc66 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.222774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa802c4b-af5f-4d9a-9398-975693f1fc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.223277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa802c4b-af5f-4d9a-9398-975693f1fc66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.237471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61c7fb-99fb-430a-ba1e-7a7ec8dc9f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.238013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61c7fb-99fb-430a-ba1e-7a7ec8dc9f6b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.238357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61c7fb-99fb-430a-ba1e-7a7ec8dc9f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.238691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61c7fb-99fb-430a-ba1e-7a7ec8dc9f6b None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.239333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61c7fb-99fb-430a-ba1e-7a7ec8dc9f6b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:00.239642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61c7fb-99fb-430a-ba1e-7a7ec8dc9f6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.241032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61c7fb-99fb-430a-ba1e-7a7ec8dc9f6b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.254698 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 356/707] 10.222.0.82 () {56 vars in 1124 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:00.255751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61c7fb-99fb-430a-ba1e-7a7ec8dc9f6b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.280637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.316142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa802c4b-af5f-4d9a-9398-975693f1fc66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.318662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa802c4b-af5f-4d9a-9398-975693f1fc66 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.413811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.422135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61c7fb-99fb-430a-ba1e-7a7ec8dc9f6b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.459295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 352/708] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:31:00.460243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa802c4b-af5f-4d9a-9398-975693f1fc66 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.477605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2a8d1-b8ea-40cb-9355-25450f653ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.478769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2a8d1-b8ea-40cb-9355-25450f653ecf None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.479258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2a8d1-b8ea-40cb-9355-25450f653ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.479549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2a8d1-b8ea-40cb-9355-25450f653ecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.495571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2a8d1-b8ea-40cb-9355-25450f653ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.497284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2a8d1-b8ea-40cb-9355-25450f653ecf None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-a3c2a8d1-b8ea-40cb-9355-25450f653ecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:00.520188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:00.521210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:00.522130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:00.522130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:00.522130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:00.522130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 357/709] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:00.522830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2a8d1-b8ea-40cb-9355-25450f653ecf None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.590816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa802c4b-af5f-4d9a-9398-975693f1fc66 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.611364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.630382 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dc231-af6d-4fe4-9165-e791755827f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.631455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dc231-af6d-4fe4-9165-e791755827f7 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.631965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dc231-af6d-4fe4-9165-e791755827f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.632438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dc231-af6d-4fe4-9165-e791755827f7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.633263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dc231-af6d-4fe4-9165-e791755827f7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.635080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dc231-af6d-4fe4-9165-e791755827f7 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-194dc231-af6d-4fe4-9165-e791755827f7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:00.647682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:00.648756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:31:00.649621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:00.651924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2a8d1-b8ea-40cb-9355-25450f653ecf None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.657547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 353/710] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:00.658827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dc231-af6d-4fe4-9165-e791755827f7 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.690164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad8513-2769-4ed0-a94a-3bf0460602d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.690825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad8513-2769-4ed0-a94a-3bf0460602d6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.691110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad8513-2769-4ed0-a94a-3bf0460602d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.691384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad8513-2769-4ed0-a94a-3bf0460602d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.693820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.703064 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad8513-2769-4ed0-a94a-3bf0460602d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.705102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad8513-2769-4ed0-a94a-3bf0460602d6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-f8ad8513-2769-4ed0-a94a-3bf0460602d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:00.720865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:00.722283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:00.723235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:00.723235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:00.723235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:00.726644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 358/711] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:00.727524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad8513-2769-4ed0-a94a-3bf0460602d6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.774147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dc231-af6d-4fe4-9165-e791755827f7 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.814781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cc991-c5d3-4016-ab89-072a229145e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.815642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cc991-c5d3-4016-ab89-072a229145e6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.815923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cc991-c5d3-4016-ab89-072a229145e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.816195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cc991-c5d3-4016-ab89-072a229145e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.816849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cc991-c5d3-4016-ab89-072a229145e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:00.817153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cc991-c5d3-4016-ab89-072a229145e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.818582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cc991-c5d3-4016-ab89-072a229145e6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.835912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 354/712] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:00.836791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cc991-c5d3-4016-ab89-072a229145e6 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.847376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad8513-2769-4ed0-a94a-3bf0460602d6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.868630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.888734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df2c60-8465-4411-9a54-066d5d31c5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.889604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df2c60-8465-4411-9a54-066d5d31c5bd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.890083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df2c60-8465-4411-9a54-066d5d31c5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.890529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df2c60-8465-4411-9a54-066d5d31c5bd None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.891449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df2c60-8465-4411-9a54-066d5d31c5bd None admin] RBAC: Adding query filter params (name=network) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:00.892012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df2c60-8465-4411-9a54-066d5d31c5bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.893558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df2c60-8465-4411-9a54-066d5d31c5bd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:00.899253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 359/713] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:00.900319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df2c60-8465-4411-9a54-066d5d31c5bd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.940025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.949320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cc991-c5d3-4016-ab89-072a229145e6 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.983680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57891b6-e5a3-48af-beb7-e728a132d907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.984298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57891b6-e5a3-48af-beb7-e728a132d907 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.985123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57891b6-e5a3-48af-beb7-e728a132d907 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.985392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57891b6-e5a3-48af-beb7-e728a132d907 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.986237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57891b6-e5a3-48af-beb7-e728a132d907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:00.986570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57891b6-e5a3-48af-beb7-e728a132d907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:00.988656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57891b6-e5a3-48af-beb7-e728a132d907 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:01.008193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 355/714] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:01.009162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57891b6-e5a3-48af-beb7-e728a132d907 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:01.024305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df2c60-8465-4411-9a54-066d5d31c5bd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:01.064370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77150-4286-4c5d-8d70-12fb15e2453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:01.065234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77150-4286-4c5d-8d70-12fb15e2453f None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.065666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77150-4286-4c5d-8d70-12fb15e2453f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.066113 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77150-4286-4c5d-8d70-12fb15e2453f None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.067041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d77150-4286-4c5d-8d70-12fb15e2453f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:01.068745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d77150-4286-4c5d-8d70-12fb15e2453f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:01.114327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57891b6-e5a3-48af-beb7-e728a132d907 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:01.155333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5bdb1-a55b-4201-9786-4eb53c8aeba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:01.157305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5bdb1-a55b-4201-9786-4eb53c8aeba4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.157305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5bdb1-a55b-4201-9786-4eb53c8aeba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.157305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5bdb1-a55b-4201-9786-4eb53c8aeba4 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.157305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5bdb1-a55b-4201-9786-4eb53c8aeba4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:01.157718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5bdb1-a55b-4201-9786-4eb53c8aeba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:01.159143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5bdb1-a55b-4201-9786-4eb53c8aeba4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:01.170990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 356/715] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:01.170990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5bdb1-a55b-4201-9786-4eb53c8aeba4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:01.194978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 360/716] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:31:00 2022] POST /identity/v3/users => generated 254 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:01.200494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:01.333436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5bdb1-a55b-4201-9786-4eb53c8aeba4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:01.395990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0caa8-1af3-4d91-9ea1-4c2a36b4949d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:01.396639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0caa8-1af3-4d91-9ea1-4c2a36b4949d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.396915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0caa8-1af3-4d91-9ea1-4c2a36b4949d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.397188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0caa8-1af3-4d91-9ea1-4c2a36b4949d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.412018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0caa8-1af3-4d91-9ea1-4c2a36b4949d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:01.413493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0caa8-1af3-4d91-9ea1-4c2a36b4949d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-82e0caa8-1af3-4d91-9ea1-4c2a36b4949d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:01.426733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:01.427707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:01.428630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:01.428630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:01.428630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:01.428630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:01.428630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:01.428630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:01.428630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:01.428630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:01.428630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 357/717] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:01.434094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77150-4286-4c5d-8d70-12fb15e2453f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:01.489915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:01.508347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404af75-0048-4ab9-975a-b6e791237906 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.508781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404af75-0048-4ab9-975a-b6e791237906 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.509202 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404af75-0048-4ab9-975a-b6e791237906 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.511660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 358/718] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:01.518102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01c1ac-7e51-493b-99a5-d3fcf49d5289 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.518537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01c1ac-7e51-493b-99a5-d3fcf49d5289 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.520210 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01c1ac-7e51-493b-99a5-d3fcf49d5289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.554681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-0c01c1ac-7e51-493b-99a5-d3fcf49d5289 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:01.555576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0c01c1ac-7e51-493b-99a5-d3fcf49d5289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:01.578062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77150-4286-4c5d-8d70-12fb15e2453f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:01.622758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32335673-a5c2-4b21-820d-e66518d2aa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:01.623308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32335673-a5c2-4b21-820d-e66518d2aa1c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.623595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32335673-a5c2-4b21-820d-e66518d2aa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.623871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32335673-a5c2-4b21-820d-e66518d2aa1c None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.624584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32335673-a5c2-4b21-820d-e66518d2aa1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:01.624895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32335673-a5c2-4b21-820d-e66518d2aa1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:01.626405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32335673-a5c2-4b21-820d-e66518d2aa1c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:01.640261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 361/719] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:01.649182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32335673-a5c2-4b21-820d-e66518d2aa1c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:01.667240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 359/720] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:31:01.763659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:01.769112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32335673-a5c2-4b21-820d-e66518d2aa1c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:01.808871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c02b0f-9af3-4478-9ce5-983718d8f82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:01.809489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c02b0f-9af3-4478-9ce5-983718d8f82c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:01.809773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c02b0f-9af3-4478-9ce5-983718d8f82c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:01.810051 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c02b0f-9af3-4478-9ce5-983718d8f82c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:01.810831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c02b0f-9af3-4478-9ce5-983718d8f82c None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=0f34fe3fe912498b90dbf950e7fc999d) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:01.811143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c02b0f-9af3-4478-9ce5-983718d8f82c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:01.812642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c02b0f-9af3-4478-9ce5-983718d8f82c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:01.822982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 362/721] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:01 2022] GET /identity/v3/role_assignments?user.id=0f34fe3fe912498b90dbf950e7fc999d&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9e904216086e4a06abf9216b0c863f89 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:02.054528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01c1ac-7e51-493b-99a5-d3fcf49d5289 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.058801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:02.194522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01c1ac-7e51-493b-99a5-d3fcf49d5289 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.251382 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb84ad-4460-4292-b4c6-bf5bbc014e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.252059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb84ad-4460-4292-b4c6-bf5bbc014e52 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.252346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb84ad-4460-4292-b4c6-bf5bbc014e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.252625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb84ad-4460-4292-b4c6-bf5bbc014e52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.265953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb84ad-4460-4292-b4c6-bf5bbc014e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-d8eb84ad-4460-4292-b4c6-bf5bbc014e52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:02.271487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:02.272464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:02.273365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 360/722] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:02.280325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c02b0f-9af3-4478-9ce5-983718d8f82c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.414591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c02b0f-9af3-4478-9ce5-983718d8f82c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.454696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d449f7-a7c5-4a71-b434-4c9895c7116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.455321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d449f7-a7c5-4a71-b434-4c9895c7116b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.455605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d449f7-a7c5-4a71-b434-4c9895c7116b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.455881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d449f7-a7c5-4a71-b434-4c9895c7116b None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.456564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d449f7-a7c5-4a71-b434-4c9895c7116b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:02.456872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d449f7-a7c5-4a71-b434-4c9895c7116b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.458273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d449f7-a7c5-4a71-b434-4c9895c7116b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.467206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 363/723] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:31:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:02.473737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb84ad-4460-4292-b4c6-bf5bbc014e52 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.568458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9489a-e44c-4393-9b2f-af74d7e2ea2d None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.568861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9489a-e44c-4393-9b2f-af74d7e2ea2d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.569349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9489a-e44c-4393-9b2f-af74d7e2ea2d None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.571825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 364/724] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:02.582713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:02.589267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb0d3-1457-4893-990b-8477d3a90852 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.589558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb0d3-1457-4893-990b-8477d3a90852 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.589841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb0d3-1457-4893-990b-8477d3a90852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.623420 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7b9fb0d3-1457-4893-990b-8477d3a90852 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:02.624660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7b9fb0d3-1457-4893-990b-8477d3a90852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:02.625835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb84ad-4460-4292-b4c6-bf5bbc014e52 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.667936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfbb77-5416-4d79-abcc-a6485680c684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.669351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfbb77-5416-4d79-abcc-a6485680c684 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.670665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfbb77-5416-4d79-abcc-a6485680c684 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.674009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfbb77-5416-4d79-abcc-a6485680c684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.674898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:02.687583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfbb77-5416-4d79-abcc-a6485680c684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.693327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfbb77-5416-4d79-abcc-a6485680c684 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-26bfbb77-5416-4d79-abcc-a6485680c684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:02.706724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:02.707872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:02.708803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:02.708803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:02.708803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:02.710964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 361/725] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:02.716834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfbb77-5416-4d79-abcc-a6485680c684 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:02.743688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 365/726] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:31:02.829104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfbb77-5416-4d79-abcc-a6485680c684 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:02.850535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:02.874260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f643f-434b-4b36-ae85-cbe135a4a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:02.875806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f643f-434b-4b36-ae85-cbe135a4a532 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:02.876423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f643f-434b-4b36-ae85-cbe135a4a532 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:02.876903 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f643f-434b-4b36-ae85-cbe135a4a532 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:02.877819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f643f-434b-4b36-ae85-cbe135a4a532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:02.878341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f643f-434b-4b36-ae85-cbe135a4a532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:02.879983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f643f-434b-4b36-ae85-cbe135a4a532 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:02.899697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 362/727] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:02.905636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb0d3-1457-4893-990b-8477d3a90852 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.099903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb0d3-1457-4893-990b-8477d3a90852 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.140126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f643f-434b-4b36-ae85-cbe135a4a532 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.145847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549c053-50fd-43b3-afc5-b2ff3e21c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.146494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549c053-50fd-43b3-afc5-b2ff3e21c745 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.146811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549c053-50fd-43b3-afc5-b2ff3e21c745 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.147090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549c053-50fd-43b3-afc5-b2ff3e21c745 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.155902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549c053-50fd-43b3-afc5-b2ff3e21c745 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.157814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549c053-50fd-43b3-afc5-b2ff3e21c745 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-1549c053-50fd-43b3-afc5-b2ff3e21c745 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:03.170954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:03.172593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:31:03.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:03.176320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 366/728] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:03.183726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549c053-50fd-43b3-afc5-b2ff3e21c745 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.222920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:03.323738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f643f-434b-4b36-ae85-cbe135a4a532 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.378486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00304c1-11ed-400a-a28d-5a9348e63acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.379374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00304c1-11ed-400a-a28d-5a9348e63acf None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.379812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00304c1-11ed-400a-a28d-5a9348e63acf None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.381065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00304c1-11ed-400a-a28d-5a9348e63acf None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.381870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00304c1-11ed-400a-a28d-5a9348e63acf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.384398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00304c1-11ed-400a-a28d-5a9348e63acf None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.411680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549c053-50fd-43b3-afc5-b2ff3e21c745 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.449499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aef1f6-1e20-4c19-8a7a-a2503a3fb047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.450145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aef1f6-1e20-4c19-8a7a-a2503a3fb047 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.450639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aef1f6-1e20-4c19-8a7a-a2503a3fb047 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.451060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aef1f6-1e20-4c19-8a7a-a2503a3fb047 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.451962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aef1f6-1e20-4c19-8a7a-a2503a3fb047 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:03.452416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aef1f6-1e20-4c19-8a7a-a2503a3fb047 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.454219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aef1f6-1e20-4c19-8a7a-a2503a3fb047 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.474467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 367/729] 10.222.0.82 () {56 vars in 1135 bytes} [Wed Aug 24 03:31:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:03.483711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aef1f6-1e20-4c19-8a7a-a2503a3fb047 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.499336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 363/730] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:31:03 2022] POST /identity/v3/services => generated 251 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:03.590308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:03.626138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aef1f6-1e20-4c19-8a7a-a2503a3fb047 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.685641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf604b-4f1a-46f4-b799-82817206d767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:03.686259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf604b-4f1a-46f4-b799-82817206d767 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.686655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf604b-4f1a-46f4-b799-82817206d767 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.686936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf604b-4f1a-46f4-b799-82817206d767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.783753 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf604b-4f1a-46f4-b799-82817206d767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:03.785264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf604b-4f1a-46f4-b799-82817206d767 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:03.828739 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9039e-9897-491f-8c38-d2212a5bcf4d None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.829930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9039e-9897-491f-8c38-d2212a5bcf4d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.830416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9039e-9897-491f-8c38-d2212a5bcf4d None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.836456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 364/731] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:03.838718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f892b-2969-4337-a643-0510724fec1c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.839386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f892b-2969-4337-a643-0510724fec1c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.840220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f892b-2969-4337-a643-0510724fec1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.862563 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-bf9f892b-2969-4337-a643-0510724fec1c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:03.864131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bf9f892b-2969-4337-a643-0510724fec1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:03.895886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:03.910800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:03.942697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 365/732] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:31:03.944324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f1ace-cca1-4e8f-a3bb-8a4a5328b732 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.944800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f1ace-cca1-4e8f-a3bb-8a4a5328b732 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.944939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 368/733] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:31:03.945405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f1ace-cca1-4e8f-a3bb-8a4a5328b732 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.949992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 366/734] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:03.950645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf604b-4f1a-46f4-b799-82817206d767 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.956926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363f4cd-4308-48a7-ab06-5d238f635aee None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.957501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363f4cd-4308-48a7-ab06-5d238f635aee None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.957986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363f4cd-4308-48a7-ab06-5d238f635aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.059560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-f363f4cd-4308-48a7-ab06-5d238f635aee None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:04.060780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f363f4cd-4308-48a7-ab06-5d238f635aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:04.072506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf604b-4f1a-46f4-b799-82817206d767 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.114691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fb8f8-7c58-415f-9ea5-51a805617329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.115338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fb8f8-7c58-415f-9ea5-51a805617329 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.115629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fb8f8-7c58-415f-9ea5-51a805617329 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.115913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fb8f8-7c58-415f-9ea5-51a805617329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.134002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fb8f8-7c58-415f-9ea5-51a805617329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.135658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fb8f8-7c58-415f-9ea5-51a805617329 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-df8fb8f8-7c58-415f-9ea5-51a805617329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:04.152978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:04.153984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:04.154901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:04.154901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:04.154901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:04.154901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 369/735] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:04.161767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fb8f8-7c58-415f-9ea5-51a805617329 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.185526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 367/736] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:31:04.200581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:04.268939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363f4cd-4308-48a7-ab06-5d238f635aee None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.326984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fb8f8-7c58-415f-9ea5-51a805617329 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.400362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6db051-f1c8-46bd-9810-b468143a5963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.401693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6db051-f1c8-46bd-9810-b468143a5963 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.402342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6db051-f1c8-46bd-9810-b468143a5963 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.403044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6db051-f1c8-46bd-9810-b468143a5963 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.404473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6db051-f1c8-46bd-9810-b468143a5963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:04.405188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6db051-f1c8-46bd-9810-b468143a5963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.408838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6db051-f1c8-46bd-9810-b468143a5963 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.423342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:04.436121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 370/737] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:04.438277 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07e3c-2cd8-409f-9b32-4d341599d860 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.438829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363f4cd-4308-48a7-ab06-5d238f635aee None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.438999 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07e3c-2cd8-409f-9b32-4d341599d860 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.439440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07e3c-2cd8-409f-9b32-4d341599d860 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.441383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 371/738] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:04.442563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07e3c-2cd8-409f-9b32-4d341599d860 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.493355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511f236-ebfd-495b-9d97-d2e79e6f689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.493896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511f236-ebfd-495b-9d97-d2e79e6f689f None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.494176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511f236-ebfd-495b-9d97-d2e79e6f689f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.494447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511f236-ebfd-495b-9d97-d2e79e6f689f None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.495121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511f236-ebfd-495b-9d97-d2e79e6f689f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.497001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511f236-ebfd-495b-9d97-d2e79e6f689f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.540668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:04.562184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07e3c-2cd8-409f-9b32-4d341599d860 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.585405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 368/739] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:31:04 2022] POST /identity/v3/services => generated 240 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:04.586979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc79d4-0445-444e-bd3e-1ca7cdac758e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.587576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc79d4-0445-444e-bd3e-1ca7cdac758e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.588158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc79d4-0445-444e-bd3e-1ca7cdac758e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.616048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6bcc79d4-0445-444e-bd3e-1ca7cdac758e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:04.617701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6bcc79d4-0445-444e-bd3e-1ca7cdac758e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:04.623112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3491-51e4-4740-9517-1c7e3d319145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.623777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3491-51e4-4740-9517-1c7e3d319145 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.624067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3491-51e4-4740-9517-1c7e3d319145 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.624347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3491-51e4-4740-9517-1c7e3d319145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.633864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe3491-51e4-4740-9517-1c7e3d319145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.635490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe3491-51e4-4740-9517-1c7e3d319145 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-2bfe3491-51e4-4740-9517-1c7e3d319145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:04.650151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:04.651161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:04.652100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:04.652100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:04.652100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:04.652100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:04.652100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:04.652100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:04.652100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:04.652100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:04.656503 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 372/740] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:04.657565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3491-51e4-4740-9517-1c7e3d319145 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.720259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 369/741] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:04.721594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc79d4-0445-444e-bd3e-1ca7cdac758e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.790671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3491-51e4-4740-9517-1c7e3d319145 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.837246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8638df-c31f-4044-9b2d-2795dce76f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.837894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8638df-c31f-4044-9b2d-2795dce76f0b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.838452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8638df-c31f-4044-9b2d-2795dce76f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.838764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8638df-c31f-4044-9b2d-2795dce76f0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.840763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:04.843048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:04.853693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8638df-c31f-4044-9b2d-2795dce76f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:04.859081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc79d4-0445-444e-bd3e-1ca7cdac758e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-9c8638df-c31f-4044-9b2d-2795dce76f0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:04.865976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:04.866941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:04.867798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 373/742] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:04.871689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8638df-c31f-4044-9b2d-2795dce76f0b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:04.943902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0a412-2f57-4ed0-a558-c19e3414a611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.944537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0a412-2f57-4ed0-a558-c19e3414a611 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.944823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0a412-2f57-4ed0-a558-c19e3414a611 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.945101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0a412-2f57-4ed0-a558-c19e3414a611 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.945897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0a412-2f57-4ed0-a558-c19e3414a611 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:04.946213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0a412-2f57-4ed0-a558-c19e3414a611 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.947710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0a412-2f57-4ed0-a558-c19e3414a611 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.974959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 370/743] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:04.981473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0a412-2f57-4ed0-a558-c19e3414a611 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.029166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8638df-c31f-4044-9b2d-2795dce76f0b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.073000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ad72d-e52b-430e-852f-65c478db2b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.073639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ad72d-e52b-430e-852f-65c478db2b94 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.074043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ad72d-e52b-430e-852f-65c478db2b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.074424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ad72d-e52b-430e-852f-65c478db2b94 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.075246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ad72d-e52b-430e-852f-65c478db2b94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:05.075653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ad72d-e52b-430e-852f-65c478db2b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.077155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ad72d-e52b-430e-852f-65c478db2b94 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.089757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 374/744] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:05.090796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ad72d-e52b-430e-852f-65c478db2b94 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.149143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0a412-2f57-4ed0-a558-c19e3414a611 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.192315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:05.192667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120edf4-3de5-4661-a543-73bbf69bebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.193305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120edf4-3de5-4661-a543-73bbf69bebcc None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.193590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120edf4-3de5-4661-a543-73bbf69bebcc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.193867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120edf4-3de5-4661-a543-73bbf69bebcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.194648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120edf4-3de5-4661-a543-73bbf69bebcc None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=ad67f6ace29a4591bf2b8b4f119b3323) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:05.194965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120edf4-3de5-4661-a543-73bbf69bebcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.196489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120edf4-3de5-4661-a543-73bbf69bebcc None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.204887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 371/745] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:04 2022] GET /identity/v3/role_assignments?user.id=ad67f6ace29a4591bf2b8b4f119b3323&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9e904216086e4a06abf9216b0c863f89 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:05.205738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120edf4-3de5-4661-a543-73bbf69bebcc None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.224285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:05.229698 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ad72d-e52b-430e-852f-65c478db2b94 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.271295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71296339-25ee-420a-8e25-6e4470f035a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.272354 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71296339-25ee-420a-8e25-6e4470f035a5 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.272835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71296339-25ee-420a-8e25-6e4470f035a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.273306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71296339-25ee-420a-8e25-6e4470f035a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.285526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71296339-25ee-420a-8e25-6e4470f035a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-71296339-25ee-420a-8e25-6e4470f035a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:05.291470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:05.292493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:05.293383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:05.296224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 375/746] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:05.302857 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71296339-25ee-420a-8e25-6e4470f035a5 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.368700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120edf4-3de5-4661-a543-73bbf69bebcc None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.431239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5259c41-4ed3-4f27-a5e9-7e044a6ed023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.433023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5259c41-4ed3-4f27-a5e9-7e044a6ed023 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.433791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5259c41-4ed3-4f27-a5e9-7e044a6ed023 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.434283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5259c41-4ed3-4f27-a5e9-7e044a6ed023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.448717 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5259c41-4ed3-4f27-a5e9-7e044a6ed023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.450658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5259c41-4ed3-4f27-a5e9-7e044a6ed023 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-c5259c41-4ed3-4f27-a5e9-7e044a6ed023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:05.468466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:05.470190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:05.471184 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:05.471184 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:05.471184 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:05.473853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 372/747] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:05.479240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5259c41-4ed3-4f27-a5e9-7e044a6ed023 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.534530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71296339-25ee-420a-8e25-6e4470f035a5 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.564217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:05.580102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:05.607100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c5e2-621b-4988-9637-1e78fe44c8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.607738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c5e2-621b-4988-9637-1e78fe44c8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.608017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c5e2-621b-4988-9637-1e78fe44c8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.608304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c5e2-621b-4988-9637-1e78fe44c8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.608982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4c5e2-621b-4988-9637-1e78fe44c8d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:05.609290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4c5e2-621b-4988-9637-1e78fe44c8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.610939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4c5e2-621b-4988-9637-1e78fe44c8d2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.636944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5259c41-4ed3-4f27-a5e9-7e044a6ed023 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.641382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 376/748] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:31:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:05.648705 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c5e2-621b-4988-9637-1e78fe44c8d2 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.685963 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946fe57-fbb2-45f8-b86e-cd412e22cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.686939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946fe57-fbb2-45f8-b86e-cd412e22cc2e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.687417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946fe57-fbb2-45f8-b86e-cd412e22cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.687972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946fe57-fbb2-45f8-b86e-cd412e22cc2e None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.688800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946fe57-fbb2-45f8-b86e-cd412e22cc2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:05.689116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946fe57-fbb2-45f8-b86e-cd412e22cc2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:05.691664 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946fe57-fbb2-45f8-b86e-cd412e22cc2e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:05.715353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 373/749] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:05.721885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946fe57-fbb2-45f8-b86e-cd412e22cc2e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:05.898950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c5e2-621b-4988-9637-1e78fe44c8d2 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.919427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946fe57-fbb2-45f8-b86e-cd412e22cc2e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:05.971941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:05.974294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90672387-1024-40d4-8288-6091f14aabd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.975517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90672387-1024-40d4-8288-6091f14aabd1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.977350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90672387-1024-40d4-8288-6091f14aabd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.979561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90672387-1024-40d4-8288-6091f14aabd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:05.997264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d3c5b-239b-4384-8589-d20206042f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:05.998207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3c5b-239b-4384-8589-d20206042f40 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:05.998557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3c5b-239b-4384-8589-d20206042f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:05.998812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d3c5b-239b-4384-8589-d20206042f40 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.007434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90672387-1024-40d4-8288-6091f14aabd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.010197 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90672387-1024-40d4-8288-6091f14aabd1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.013789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3c5b-239b-4384-8589-d20206042f40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.015502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d3c5b-239b-4384-8589-d20206042f40 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-5c9d3c5b-239b-4384-8589-d20206042f40 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:06.034750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:06.035819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:06.036793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 374/750] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:31:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:06.038261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb6462-99b0-4643-81d8-10863b1041dd None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.038649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb6462-99b0-4643-81d8-10863b1041dd None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.039037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb6462-99b0-4643-81d8-10863b1041dd None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-90672387-1024-40d4-8288-6091f14aabd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:06.040225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:06.041515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:06.042425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:06.042425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:06.042425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:06.044409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 377/752] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:06.044651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 375/752] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:06.045608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90672387-1024-40d4-8288-6091f14aabd1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.050558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eeb1bb-4527-425c-aa49-76910c1511f5 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.050558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eeb1bb-4527-425c-aa49-76910c1511f5 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.050558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eeb1bb-4527-425c-aa49-76910c1511f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.092045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:06.185536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-78eeb1bb-4527-425c-aa49-76910c1511f5 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:06.185751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90672387-1024-40d4-8288-6091f14aabd1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:06.186982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-78eeb1bb-4527-425c-aa49-76910c1511f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:06.232855 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30897410-44bd-4a14-b73d-e3c83a300144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:06.233588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30897410-44bd-4a14-b73d-e3c83a300144 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.233959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30897410-44bd-4a14-b73d-e3c83a300144 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.234304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30897410-44bd-4a14-b73d-e3c83a300144 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.235976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30897410-44bd-4a14-b73d-e3c83a300144 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:06.236365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30897410-44bd-4a14-b73d-e3c83a300144 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.243040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30897410-44bd-4a14-b73d-e3c83a300144 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.267157 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 378/753] 10.222.0.82 () {56 vars in 1149 bytes} [Wed Aug 24 03:31:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:06.268614 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30897410-44bd-4a14-b73d-e3c83a300144 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.287936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 376/754] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:06.287936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eeb1bb-4527-425c-aa49-76910c1511f5 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.479192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:06.498992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30897410-44bd-4a14-b73d-e3c83a300144 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:06.524779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eeb1bb-4527-425c-aa49-76910c1511f5 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:06.545879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:06.546097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f587c-02ef-46ea-85b5-7fcea0c5bde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:06.546754 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f587c-02ef-46ea-85b5-7fcea0c5bde7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.550531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f587c-02ef-46ea-85b5-7fcea0c5bde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.550531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f587c-02ef-46ea-85b5-7fcea0c5bde7 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.550531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f587c-02ef-46ea-85b5-7fcea0c5bde7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:06.550531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f587c-02ef-46ea-85b5-7fcea0c5bde7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.550531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f587c-02ef-46ea-85b5-7fcea0c5bde7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.562486 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 379/755] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:06.572086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f587c-02ef-46ea-85b5-7fcea0c5bde7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.579343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c89d3a-e6b3-4294-8013-78bab977eac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:06.580371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c89d3a-e6b3-4294-8013-78bab977eac3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.580887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c89d3a-e6b3-4294-8013-78bab977eac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.581371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c89d3a-e6b3-4294-8013-78bab977eac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.654943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c89d3a-e6b3-4294-8013-78bab977eac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.657477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c89d3a-e6b3-4294-8013-78bab977eac3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.742694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f587c-02ef-46ea-85b5-7fcea0c5bde7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:06.774195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 377/756] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:31:06.775659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c89d3a-e6b3-4294-8013-78bab977eac3 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.784580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb15f6-6b69-4bed-bb79-9990efe5a9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:06.785220 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb15f6-6b69-4bed-bb79-9990efe5a9bd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.785505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb15f6-6b69-4bed-bb79-9990efe5a9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.785782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb15f6-6b69-4bed-bb79-9990efe5a9bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.794048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:06.795981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb15f6-6b69-4bed-bb79-9990efe5a9bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.798921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb15f6-6b69-4bed-bb79-9990efe5a9bd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-a9fb15f6-6b69-4bed-bb79-9990efe5a9bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:06.810454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:06.811546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 380/757] 10.222.0.82 () {56 vars in 1093 bytes} [Wed Aug 24 03:31:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:06.813289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb15f6-6b69-4bed-bb79-9990efe5a9bd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.826336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:06.903203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c89d3a-e6b3-4294-8013-78bab977eac3 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:06.922979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb15f6-6b69-4bed-bb79-9990efe5a9bd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:06.951993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e1cce-20cf-4f3c-96fe-9e436f3454e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:06.952633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e1cce-20cf-4f3c-96fe-9e436f3454e3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.952909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e1cce-20cf-4f3c-96fe-9e436f3454e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.953745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e1cce-20cf-4f3c-96fe-9e436f3454e3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.954314 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e1cce-20cf-4f3c-96fe-9e436f3454e3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:06.955966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e1cce-20cf-4f3c-96fe-9e436f3454e3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-b08e1cce-20cf-4f3c-96fe-9e436f3454e3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:06.967682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:06.968935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:31:06.970467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:06.972063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 378/758] 10.222.0.82 () {56 vars in 1129 bytes} [Wed Aug 24 03:31:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:06.972284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705027cb-d7f3-4bef-a726-edc8f0d43811 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.972538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705027cb-d7f3-4bef-a726-edc8f0d43811 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.972820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705027cb-d7f3-4bef-a726-edc8f0d43811 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.974335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 379/759] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:06.979599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705027cb-d7f3-4bef-a726-edc8f0d43811 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.986491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da31b19-d40c-4695-a6b8-9af5be58e6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:06.987431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da31b19-d40c-4695-a6b8-9af5be58e6bb None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.987717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da31b19-d40c-4695-a6b8-9af5be58e6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.988044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da31b19-d40c-4695-a6b8-9af5be58e6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.005024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da31b19-d40c-4695-a6b8-9af5be58e6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.006815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da31b19-d40c-4695-a6b8-9af5be58e6bb None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-3da31b19-d40c-4695-a6b8-9af5be58e6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:07.023761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:07.024736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:07.025645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:07.025645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:07.025645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:07.025645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 381/760] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:07.025645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da31b19-d40c-4695-a6b8-9af5be58e6bb None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.080427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.094727 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.110995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705027cb-d7f3-4bef-a726-edc8f0d43811 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.134618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da31b19-d40c-4695-a6b8-9af5be58e6bb None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.153299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bb59-9ee7-4ef4-9957-24fa02ce888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.159861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bb59-9ee7-4ef4-9957-24fa02ce888b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.159861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bb59-9ee7-4ef4-9957-24fa02ce888b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.159861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bb59-9ee7-4ef4-9957-24fa02ce888b None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.159861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bb59-9ee7-4ef4-9957-24fa02ce888b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:07.159861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bb59-9ee7-4ef4-9957-24fa02ce888b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.159861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bb59-9ee7-4ef4-9957-24fa02ce888b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.173882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e7ace-aba7-40a2-8b40-91d6fb099ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.174530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e7ace-aba7-40a2-8b40-91d6fb099ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.174825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e7ace-aba7-40a2-8b40-91d6fb099ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.175106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e7ace-aba7-40a2-8b40-91d6fb099ff6 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.175745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e7ace-aba7-40a2-8b40-91d6fb099ff6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:07.176055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e7ace-aba7-40a2-8b40-91d6fb099ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.181459 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 380/761] 10.222.0.82 () {56 vars in 1133 bytes} [Wed Aug 24 03:31:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:07.181607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e7ace-aba7-40a2-8b40-91d6fb099ff6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.181710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874d286-30d1-4199-9db6-caab9d61d8fd None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.181710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874d286-30d1-4199-9db6-caab9d61d8fd None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.181710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874d286-30d1-4199-9db6-caab9d61d8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.188004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 382/762] 10.222.0.82 () {56 vars in 1124 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:07.189184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e7ace-aba7-40a2-8b40-91d6fb099ff6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.197524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b874d286-30d1-4199-9db6-caab9d61d8fd None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:07.198794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b874d286-30d1-4199-9db6-caab9d61d8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:07.279001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 381/763] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:07.280151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874d286-30d1-4199-9db6-caab9d61d8fd None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.325800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e7ace-aba7-40a2-8b40-91d6fb099ff6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.353788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.363350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.374452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49f390-37e9-48d8-bedb-4acd4643afdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.376587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49f390-37e9-48d8-bedb-4acd4643afdf None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.377152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49f390-37e9-48d8-bedb-4acd4643afdf None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.377440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49f390-37e9-48d8-bedb-4acd4643afdf None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.378105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49f390-37e9-48d8-bedb-4acd4643afdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:07.378597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49f390-37e9-48d8-bedb-4acd4643afdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.380428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49f390-37e9-48d8-bedb-4acd4643afdf None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.394454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874d286-30d1-4199-9db6-caab9d61d8fd None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.398724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 383/764] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:07.399939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49f390-37e9-48d8-bedb-4acd4643afdf None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.434176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bb454-cb4d-423f-8a1b-f111b1d258b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.434847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bb454-cb4d-423f-8a1b-f111b1d258b4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.435126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bb454-cb4d-423f-8a1b-f111b1d258b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.435399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bb454-cb4d-423f-8a1b-f111b1d258b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.448238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bb454-cb4d-423f-8a1b-f111b1d258b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.449713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bb454-cb4d-423f-8a1b-f111b1d258b4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-c40bb454-cb4d-423f-8a1b-f111b1d258b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:07.466167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:07.467351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:07.468265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:07.468265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:07.468265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:07.468265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 382/765] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:07.469002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1adcc-a413-4f5a-84d2-268cef24c2c2 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.469208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1adcc-a413-4f5a-84d2-268cef24c2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.469446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1adcc-a413-4f5a-84d2-268cef24c2c2 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.471009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 383/766] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:07.471637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1adcc-a413-4f5a-84d2-268cef24c2c2 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.520312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49f390-37e9-48d8-bedb-4acd4643afdf None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.558830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53b92f-30f9-4817-b4a5-8653f659982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.559362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53b92f-30f9-4817-b4a5-8653f659982d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.559672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53b92f-30f9-4817-b4a5-8653f659982d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.559920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53b92f-30f9-4817-b4a5-8653f659982d None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.561053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53b92f-30f9-4817-b4a5-8653f659982d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:07.561366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53b92f-30f9-4817-b4a5-8653f659982d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.563925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53b92f-30f9-4817-b4a5-8653f659982d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.569806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 384/767] 10.222.0.82 () {56 vars in 1124 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:07.570537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53b92f-30f9-4817-b4a5-8653f659982d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.594086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1adcc-a413-4f5a-84d2-268cef24c2c2 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.611188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.622404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.629814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6ad2f-c09a-418b-8c5d-2f78eaba0805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.631766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6ad2f-c09a-418b-8c5d-2f78eaba0805 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.631766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6ad2f-c09a-418b-8c5d-2f78eaba0805 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.631766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6ad2f-c09a-418b-8c5d-2f78eaba0805 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.637488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6ad2f-c09a-418b-8c5d-2f78eaba0805 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.638976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6ad2f-c09a-418b-8c5d-2f78eaba0805 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-38b6ad2f-c09a-418b-8c5d-2f78eaba0805 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:07.649863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:07.650939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:07.651929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:07.651929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:07.651929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:07.651929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:07.651929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:07.651929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:07.651929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:07.651929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:07.651929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 384/768] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:07.652620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e20bb-ec8e-4f1d-a16a-6e02bee5b580 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.652620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e20bb-ec8e-4f1d-a16a-6e02bee5b580 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.652620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e20bb-ec8e-4f1d-a16a-6e02bee5b580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.671283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e83e20bb-ec8e-4f1d-a16a-6e02bee5b580 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:07.674919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e83e20bb-ec8e-4f1d-a16a-6e02bee5b580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:07.677445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53b92f-30f9-4817-b4a5-8653f659982d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.720361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2d1a-37ba-41ae-8ffa-d9a976e7d2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.720974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2d1a-37ba-41ae-8ffa-d9a976e7d2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.721252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2d1a-37ba-41ae-8ffa-d9a976e7d2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.721524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2d1a-37ba-41ae-8ffa-d9a976e7d2f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.722178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2d1a-37ba-41ae-8ffa-d9a976e7d2f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:07.722481 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2d1a-37ba-41ae-8ffa-d9a976e7d2f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.723919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2d1a-37ba-41ae-8ffa-d9a976e7d2f7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.736189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 385/769] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:07.737063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2d1a-37ba-41ae-8ffa-d9a976e7d2f7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.760104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 385/770] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:07.760977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e20bb-ec8e-4f1d-a16a-6e02bee5b580 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.873469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2d1a-37ba-41ae-8ffa-d9a976e7d2f7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.889027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.894922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e20bb-ec8e-4f1d-a16a-6e02bee5b580 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.904316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.924542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc54057-7fb0-43cf-9191-58985cd05e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.925264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54057-7fb0-43cf-9191-58985cd05e47 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.925625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54057-7fb0-43cf-9191-58985cd05e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.925967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54057-7fb0-43cf-9191-58985cd05e47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.926732 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc54057-7fb0-43cf-9191-58985cd05e47 None admin] RBAC: Adding query filter params (interface=public, service_id=63d7d230470d47489e45dfff6b815eda, region_id=RegionOne) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:07.927053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc54057-7fb0-43cf-9191-58985cd05e47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.928468 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc54057-7fb0-43cf-9191-58985cd05e47 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.936166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 386/771] 10.222.0.82 () {56 vars in 1251 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/endpoints?service_id=63d7d230470d47489e45dfff6b815eda&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:07.941412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85439a2c-a228-4696-836c-a70b0ba57a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.942098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85439a2c-a228-4696-836c-a70b0ba57a35 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.942400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85439a2c-a228-4696-836c-a70b0ba57a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.942711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85439a2c-a228-4696-836c-a70b0ba57a35 None admin] PATH_INFO: `/v3/services/network` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.942889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc54057-7fb0-43cf-9191-58985cd05e47 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:07.943402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85439a2c-a228-4696-836c-a70b0ba57a35 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.944794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85439a2c-a228-4696-836c-a70b0ba57a35 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-85439a2c-a228-4696-836c-a70b0ba57a35 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:07.955084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:07.959141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:07.960024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 386/772] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:07.960849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85439a2c-a228-4696-836c-a70b0ba57a35 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.083821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc54057-7fb0-43cf-9191-58985cd05e47 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.088203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85439a2c-a228-4696-836c-a70b0ba57a35 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.124750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab069f2-edfa-4a0f-aed4-038edb72ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.126562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab069f2-edfa-4a0f-aed4-038edb72ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.126562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab069f2-edfa-4a0f-aed4-038edb72ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.126562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab069f2-edfa-4a0f-aed4-038edb72ebe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.127202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab069f2-edfa-4a0f-aed4-038edb72ebe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:08.127506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab069f2-edfa-4a0f-aed4-038edb72ebe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.128885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9a2ab-e937-497d-82e8-0f25a9776a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.129018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab069f2-edfa-4a0f-aed4-038edb72ebe2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.129526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9a2ab-e937-497d-82e8-0f25a9776a94 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.129808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9a2ab-e937-497d-82e8-0f25a9776a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.130084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9a2ab-e937-497d-82e8-0f25a9776a94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.137819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9a2ab-e937-497d-82e8-0f25a9776a94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.139631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9a2ab-e937-497d-82e8-0f25a9776a94 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.142610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 387/773] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:08.143460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab069f2-edfa-4a0f-aed4-038edb72ebe2 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-3ea9a2ab-e937-497d-82e8-0f25a9776a94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:08.157109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:08.158229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 387/774] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:08.159206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9a2ab-e937-497d-82e8-0f25a9776a94 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.192419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:08.196259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:08.271452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab069f2-edfa-4a0f-aed4-038edb72ebe2 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.272076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9a2ab-e937-497d-82e8-0f25a9776a94 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.313998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e7c5f-8a8c-4bbf-a750-4fc4c301ac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.314686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e7c5f-8a8c-4bbf-a750-4fc4c301ac17 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.314974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e7c5f-8a8c-4bbf-a750-4fc4c301ac17 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.315247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e7c5f-8a8c-4bbf-a750-4fc4c301ac17 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.315871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26336cf9-8c9c-49db-9a83-14f1aa275245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.315978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e7c5f-8a8c-4bbf-a750-4fc4c301ac17 None admin] RBAC: Adding query filter params (name=network) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:08.316183 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e7c5f-8a8c-4bbf-a750-4fc4c301ac17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.316485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26336cf9-8c9c-49db-9a83-14f1aa275245 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.316768 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26336cf9-8c9c-49db-9a83-14f1aa275245 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.317042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26336cf9-8c9c-49db-9a83-14f1aa275245 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.317570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e7c5f-8a8c-4bbf-a750-4fc4c301ac17 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.317673 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26336cf9-8c9c-49db-9a83-14f1aa275245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.319046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26336cf9-8c9c-49db-9a83-14f1aa275245 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.322948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 388/775] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:08.323168 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e7c5f-8a8c-4bbf-a750-4fc4c301ac17 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.325816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 388/776] 10.222.0.82 () {56 vars in 1087 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:08.326699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26336cf9-8c9c-49db-9a83-14f1aa275245 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.455042 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:08.460110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:08.463081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e7c5f-8a8c-4bbf-a750-4fc4c301ac17 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.464808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26336cf9-8c9c-49db-9a83-14f1aa275245 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.508094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605a45-b4b7-4112-b644-cf1234b41fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.508707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd61c7a-7585-4d81-b618-6c41c06f7b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.509537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605a45-b4b7-4112-b644-cf1234b41fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.509537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605a45-b4b7-4112-b644-cf1234b41fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.509537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b605a45-b4b7-4112-b644-cf1234b41fb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.509773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd61c7a-7585-4d81-b618-6c41c06f7b8f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.509773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd61c7a-7585-4d81-b618-6c41c06f7b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.510058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd61c7a-7585-4d81-b618-6c41c06f7b8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.510775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd61c7a-7585-4d81-b618-6c41c06f7b8f None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=b14c2eb2840b46538f169c39f200639f) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:08.511088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd61c7a-7585-4d81-b618-6c41c06f7b8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.512569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd61c7a-7585-4d81-b618-6c41c06f7b8f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.519952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 389/777] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/role_assignments?user.id=b14c2eb2840b46538f169c39f200639f&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:08.520827 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd61c7a-7585-4d81-b618-6c41c06f7b8f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.539570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b605a45-b4b7-4112-b644-cf1234b41fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-0b605a45-b4b7-4112-b644-cf1234b41fb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:08.547587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:08.548645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:08.549647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 389/778] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:08.551707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605a45-b4b7-4112-b644-cf1234b41fb8 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.680353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd61c7a-7585-4d81-b618-6c41c06f7b8f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.697328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b605a45-b4b7-4112-b644-cf1234b41fb8 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.732712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c7c0e-c969-4e92-8ac3-fc14c2f1da6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.733534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c7c0e-c969-4e92-8ac3-fc14c2f1da6d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.733827 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c7c0e-c969-4e92-8ac3-fc14c2f1da6d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.734395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c7c0e-c969-4e92-8ac3-fc14c2f1da6d None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.735929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c7c0e-c969-4e92-8ac3-fc14c2f1da6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:08.736529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c7c0e-c969-4e92-8ac3-fc14c2f1da6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.738388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c7c0e-c969-4e92-8ac3-fc14c2f1da6d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.741304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffcfab-44e2-4d22-8267-f3d2b203642b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.742015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffcfab-44e2-4d22-8267-f3d2b203642b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.742303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffcfab-44e2-4d22-8267-f3d2b203642b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.742676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffcfab-44e2-4d22-8267-f3d2b203642b None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.743580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffcfab-44e2-4d22-8267-f3d2b203642b None admin] RBAC: Adding query filter params (type=network) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:08.743896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffcfab-44e2-4d22-8267-f3d2b203642b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.745802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffcfab-44e2-4d22-8267-f3d2b203642b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.756931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 390/779] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:08.756931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffcfab-44e2-4d22-8267-f3d2b203642b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.762641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 390/780] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:08.770181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c7c0e-c969-4e92-8ac3-fc14c2f1da6d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.781012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:08.807035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:08.885257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffcfab-44e2-4d22-8267-f3d2b203642b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.915074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c7c0e-c969-4e92-8ac3-fc14c2f1da6d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:08.931205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c66a57-64cf-4d2c-b268-85600780ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.931821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c66a57-64cf-4d2c-b268-85600780ac30 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.932101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c66a57-64cf-4d2c-b268-85600780ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.932699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c66a57-64cf-4d2c-b268-85600780ac30 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.936079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c66a57-64cf-4d2c-b268-85600780ac30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:08.936079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c66a57-64cf-4d2c-b268-85600780ac30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.936079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c66a57-64cf-4d2c-b268-85600780ac30 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.960648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 391/781] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:08.962664 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c66a57-64cf-4d2c-b268-85600780ac30 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:08.968584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5ae35-f9e1-487b-a79c-8f0b81016f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:08.970082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5ae35-f9e1-487b-a79c-8f0b81016f43 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:08.970465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5ae35-f9e1-487b-a79c-8f0b81016f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:08.970996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5ae35-f9e1-487b-a79c-8f0b81016f43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:08.972108 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5ae35-f9e1-487b-a79c-8f0b81016f43 None admin] RBAC: Adding query filter params (interface=public, service_id=422c01e08de14811b703083c065b984e, region_id=RegionOne) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:08.972478 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5ae35-f9e1-487b-a79c-8f0b81016f43 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:08.973956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5ae35-f9e1-487b-a79c-8f0b81016f43 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:08.980760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 391/782] 10.222.0.82 () {56 vars in 1251 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/endpoints?service_id=422c01e08de14811b703083c065b984e&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:08.981900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5ae35-f9e1-487b-a79c-8f0b81016f43 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.080236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:09.083208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:09.097360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c66a57-64cf-4d2c-b268-85600780ac30 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.097563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5ae35-f9e1-487b-a79c-8f0b81016f43 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.136034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef1e62-45b0-490c-8c25-700ab8ca0cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.137083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef1e62-45b0-490c-8c25-700ab8ca0cdb None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.137585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef1e62-45b0-490c-8c25-700ab8ca0cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.138071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef1e62-45b0-490c-8c25-700ab8ca0cdb None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/0f34fe3fe912498b90dbf950e7fc999d/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.140254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef1e62-45b0-490c-8c25-700ab8ca0cdb None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=0f34fe3fe912498b90dbf950e7fc999d, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.142455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef1e62-45b0-490c-8c25-700ab8ca0cdb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.156176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df601f5-a290-4e27-b2ec-b75d6ccf577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.156176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df601f5-a290-4e27-b2ec-b75d6ccf577c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.156176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df601f5-a290-4e27-b2ec-b75d6ccf577c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.156176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df601f5-a290-4e27-b2ec-b75d6ccf577c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.179933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df601f5-a290-4e27-b2ec-b75d6ccf577c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.182649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df601f5-a290-4e27-b2ec-b75d6ccf577c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-8df601f5-a290-4e27-b2ec-b75d6ccf577c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:09.215252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:09.217164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:09.218131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:09.218131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:09.218131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:09.220152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 392/784] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:09.221628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df601f5-a290-4e27-b2ec-b75d6ccf577c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.223277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 392/784] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:31:08 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/0f34fe3fe912498b90dbf950e7fc999d/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:09.244665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef1e62-45b0-490c-8c25-700ab8ca0cdb None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.354193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df601f5-a290-4e27-b2ec-b75d6ccf577c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.421142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffe619-4be6-42f6-87b5-ed7e9e49c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.422692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffe619-4be6-42f6-87b5-ed7e9e49c00a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.426078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffe619-4be6-42f6-87b5-ed7e9e49c00a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.426811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffe619-4be6-42f6-87b5-ed7e9e49c00a None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.428556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffe619-4be6-42f6-87b5-ed7e9e49c00a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.433915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffe619-4be6-42f6-87b5-ed7e9e49c00a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.451051 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 393/785] 10.222.0.82 () {56 vars in 1087 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/services => generated 772 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:09.520100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef1e62-45b0-490c-8c25-700ab8ca0cdb None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.570213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b682c5-8d8b-40ad-944d-d6947516d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.570997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b682c5-8d8b-40ad-944d-d6947516d18c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.571281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b682c5-8d8b-40ad-944d-d6947516d18c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.571564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b682c5-8d8b-40ad-944d-d6947516d18c None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.572239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b682c5-8d8b-40ad-944d-d6947516d18c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:09.572556 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b682c5-8d8b-40ad-944d-d6947516d18c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.573970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b682c5-8d8b-40ad-944d-d6947516d18c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.586035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:09.590969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 393/786] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:09.598122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffe619-4be6-42f6-87b5-ed7e9e49c00a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.601500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:09.745768 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffe619-4be6-42f6-87b5-ed7e9e49c00a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:09.791616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478679f0-447f-4d9c-b927-c22ded6a220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:09.792623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478679f0-447f-4d9c-b927-c22ded6a220c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.793132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478679f0-447f-4d9c-b927-c22ded6a220c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.793626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478679f0-447f-4d9c-b927-c22ded6a220c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.803535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478679f0-447f-4d9c-b927-c22ded6a220c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:09.805382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478679f0-447f-4d9c-b927-c22ded6a220c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-478679f0-447f-4d9c-b927-c22ded6a220c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:09.818403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:09.819475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:31:09.820450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:09.823050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 394/787] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:09.829297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b682c5-8d8b-40ad-944d-d6947516d18c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.977418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b682c5-8d8b-40ad-944d-d6947516d18c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.063373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f46aa-4d8a-4e8a-bf79-4b452a337a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.064241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f46aa-4d8a-4e8a-bf79-4b452a337a77 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.064475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f46aa-4d8a-4e8a-bf79-4b452a337a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.064728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f46aa-4d8a-4e8a-bf79-4b452a337a77 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.065492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f46aa-4d8a-4e8a-bf79-4b452a337a77 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:10.065805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f46aa-4d8a-4e8a-bf79-4b452a337a77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.067302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f46aa-4d8a-4e8a-bf79-4b452a337a77 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.097845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 394/788] 10.222.0.82 () {56 vars in 1135 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:10.103823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478679f0-447f-4d9c-b927-c22ded6a220c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.178972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:10.226001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478679f0-447f-4d9c-b927-c22ded6a220c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.269601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19fca-1a49-41c8-8cf6-bcadbb7d3185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.270324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19fca-1a49-41c8-8cf6-bcadbb7d3185 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.270566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19fca-1a49-41c8-8cf6-bcadbb7d3185 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.270851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19fca-1a49-41c8-8cf6-bcadbb7d3185 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.285436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff19fca-1a49-41c8-8cf6-bcadbb7d3185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.287023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff19fca-1a49-41c8-8cf6-bcadbb7d3185 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-6ff19fca-1a49-41c8-8cf6-bcadbb7d3185 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:10.304015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:10.305135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:10.307049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:10.307049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:10.307049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:10.307049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 395/789] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:10.311003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f46aa-4d8a-4e8a-bf79-4b452a337a77 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.407385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:10.467818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f46aa-4d8a-4e8a-bf79-4b452a337a77 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.521808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb532df-c440-4f67-8d49-dfec0cc74a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.522452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb532df-c440-4f67-8d49-dfec0cc74a9c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.522820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb532df-c440-4f67-8d49-dfec0cc74a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.527008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb532df-c440-4f67-8d49-dfec0cc74a9c None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.527751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb532df-c440-4f67-8d49-dfec0cc74a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:10.528075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb532df-c440-4f67-8d49-dfec0cc74a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.529505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb532df-c440-4f67-8d49-dfec0cc74a9c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.546482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 395/790] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:10.553489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19fca-1a49-41c8-8cf6-bcadbb7d3185 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:10.727035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d731d-6af4-4954-a9b6-5872f3c271ad None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.727420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d731d-6af4-4954-a9b6-5872f3c271ad None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.727596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d731d-6af4-4954-a9b6-5872f3c271ad None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.729311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 396/791] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:10.740097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a0be4-486e-4662-a773-063cfe40299a None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.740408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a0be4-486e-4662-a773-063cfe40299a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.740648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a0be4-486e-4662-a773-063cfe40299a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.773592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-722a0be4-486e-4662-a773-063cfe40299a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:10.774761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-722a0be4-486e-4662-a773-063cfe40299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:10.925064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 397/792] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:10.930434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19fca-1a49-41c8-8cf6-bcadbb7d3185 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.937776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b70f2-36c9-44e9-a60d-a14a595a5b12 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.939347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b70f2-36c9-44e9-a60d-a14a595a5b12 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.939787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b70f2-36c9-44e9-a60d-a14a595a5b12 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.942410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 398/793] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:10.948644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050861f-a5bd-455c-9d8b-7435f2087086 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.948939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050861f-a5bd-455c-9d8b-7435f2087086 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.949237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050861f-a5bd-455c-9d8b-7435f2087086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.964651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:10.990343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a050861f-a5bd-455c-9d8b-7435f2087086 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:10.991667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a050861f-a5bd-455c-9d8b-7435f2087086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:11.018596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3a1d8-b9dc-4870-8f71-a37b2ccf3324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.018596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3a1d8-b9dc-4870-8f71-a37b2ccf3324 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.018596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3a1d8-b9dc-4870-8f71-a37b2ccf3324 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.018596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3a1d8-b9dc-4870-8f71-a37b2ccf3324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.035430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3a1d8-b9dc-4870-8f71-a37b2ccf3324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.038390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3a1d8-b9dc-4870-8f71-a37b2ccf3324 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-9fc3a1d8-b9dc-4870-8f71-a37b2ccf3324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:11.062071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:11.064269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:11.065202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:11.065202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:11.065202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:11.065202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:11.065202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:11.065202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:11.065202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:11.065202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:11.065202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 396/794] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 511 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:11.071541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3a1d8-b9dc-4870-8f71-a37b2ccf3324 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.077720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:11.113371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 399/795] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:11.205001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3a1d8-b9dc-4870-8f71-a37b2ccf3324 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.239593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050861f-a5bd-455c-9d8b-7435f2087086 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.256324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c6782-f880-401a-b50a-2370fb3642bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.257243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c6782-f880-401a-b50a-2370fb3642bb None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.257665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c6782-f880-401a-b50a-2370fb3642bb None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.258076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c6782-f880-401a-b50a-2370fb3642bb None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.258961 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c6782-f880-401a-b50a-2370fb3642bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:11.259571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c6782-f880-401a-b50a-2370fb3642bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.261254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c6782-f880-401a-b50a-2370fb3642bb None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.276605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 397/796] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:11.282983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c6782-f880-401a-b50a-2370fb3642bb None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.358956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:11.414069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050861f-a5bd-455c-9d8b-7435f2087086 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.467740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c6782-f880-401a-b50a-2370fb3642bb None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.477927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dfaa3-f0cc-4182-a5ca-6526b5444be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.478891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dfaa3-f0cc-4182-a5ca-6526b5444be0 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.479345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dfaa3-f0cc-4182-a5ca-6526b5444be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.479786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dfaa3-f0cc-4182-a5ca-6526b5444be0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.481097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11dfaa3-f0cc-4182-a5ca-6526b5444be0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.483934 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11dfaa3-f0cc-4182-a5ca-6526b5444be0 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-e11dfaa3-f0cc-4182-a5ca-6526b5444be0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:11.498868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:11.500124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:31:11.501655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:11.504456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 400/797] 10.222.0.82 () {56 vars in 1129 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:11.505460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dfaa3-f0cc-4182-a5ca-6526b5444be0 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.522656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357209d-567b-49bd-8f41-11ad433fa067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.523811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357209d-567b-49bd-8f41-11ad433fa067 None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.524160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357209d-567b-49bd-8f41-11ad433fa067 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.524717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357209d-567b-49bd-8f41-11ad433fa067 None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/ad67f6ace29a4591bf2b8b4f119b3323/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.527429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357209d-567b-49bd-8f41-11ad433fa067 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=ad67f6ace29a4591bf2b8b4f119b3323, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.529050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357209d-567b-49bd-8f41-11ad433fa067 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.554121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:11.617747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dfaa3-f0cc-4182-a5ca-6526b5444be0 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.636543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 398/798] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:31:11 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/ad67f6ace29a4591bf2b8b4f119b3323/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:11.638452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357209d-567b-49bd-8f41-11ad433fa067 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.684702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6cb86-1b4b-4327-8fa2-fa125f5faa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.685380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6cb86-1b4b-4327-8fa2-fa125f5faa0e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.685668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6cb86-1b4b-4327-8fa2-fa125f5faa0e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.686572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6cb86-1b4b-4327-8fa2-fa125f5faa0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.715836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6cb86-1b4b-4327-8fa2-fa125f5faa0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-93e6cb86-1b4b-4327-8fa2-fa125f5faa0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:11.733605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:11.735194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:11.736242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 401/799] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:11.741269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6cb86-1b4b-4327-8fa2-fa125f5faa0e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.834861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:11.845347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357209d-567b-49bd-8f41-11ad433fa067 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.892905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13528cd8-7d46-487f-9400-8edbea2dd0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.893849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13528cd8-7d46-487f-9400-8edbea2dd0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.894400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13528cd8-7d46-487f-9400-8edbea2dd0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.894896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13528cd8-7d46-487f-9400-8edbea2dd0d0 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.895772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13528cd8-7d46-487f-9400-8edbea2dd0d0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:11.896274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13528cd8-7d46-487f-9400-8edbea2dd0d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.897850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13528cd8-7d46-487f-9400-8edbea2dd0d0 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.903138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 399/800] 10.222.0.82 () {56 vars in 1124 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:11.903632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6cb86-1b4b-4327-8fa2-fa125f5faa0e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:11.905013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cf62c-ed92-4843-98a0-5a4fe2d44872 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.905309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:11.905720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cf62c-ed92-4843-98a0-5a4fe2d44872 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.906199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cf62c-ed92-4843-98a0-5a4fe2d44872 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.912728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 400/801] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:11.913823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cf62c-ed92-4843-98a0-5a4fe2d44872 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:11.949178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae892f3-724e-4f23-b15d-f55574ceaf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:11.949906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae892f3-724e-4f23-b15d-f55574ceaf49 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.950277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae892f3-724e-4f23-b15d-f55574ceaf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.950582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae892f3-724e-4f23-b15d-f55574ceaf49 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.951370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae892f3-724e-4f23-b15d-f55574ceaf49 None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:11.951684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae892f3-724e-4f23-b15d-f55574ceaf49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:11.953194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae892f3-724e-4f23-b15d-f55574ceaf49 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:11.970599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 402/802] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:11.970599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30b10b-dc9a-4bf2-9cf1-817a392aeaf9 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:11.970599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30b10b-dc9a-4bf2-9cf1-817a392aeaf9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:11.970599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30b10b-dc9a-4bf2-9cf1-817a392aeaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:11.990694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-de30b10b-dc9a-4bf2-9cf1-817a392aeaf9 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:11.994140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-de30b10b-dc9a-4bf2-9cf1-817a392aeaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:12.067586 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cf62c-ed92-4843-98a0-5a4fe2d44872 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.102532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 403/803] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:12.102532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30b10b-dc9a-4bf2-9cf1-817a392aeaf9 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.128943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fcd9c-0cf7-468b-ab7e-a96da46eb8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.129612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fcd9c-0cf7-468b-ab7e-a96da46eb8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.129888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fcd9c-0cf7-468b-ab7e-a96da46eb8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.130156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fcd9c-0cf7-468b-ab7e-a96da46eb8d5 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.130887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fcd9c-0cf7-468b-ab7e-a96da46eb8d5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:12.131186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fcd9c-0cf7-468b-ab7e-a96da46eb8d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.132716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fcd9c-0cf7-468b-ab7e-a96da46eb8d5 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.138229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 401/804] 10.222.0.82 () {56 vars in 1124 bytes} [Wed Aug 24 03:31:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:12.139810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07832ac-2097-4670-9d97-954ad02c9b4b None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.140140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07832ac-2097-4670-9d97-954ad02c9b4b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.140418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07832ac-2097-4670-9d97-954ad02c9b4b None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.142657 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 402/805] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:12.144138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07832ac-2097-4670-9d97-954ad02c9b4b None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.171255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:12.204176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:12.207152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30b10b-dc9a-4bf2-9cf1-817a392aeaf9 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.244477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9858a7-dffa-4dca-8648-2e43743884b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.245133 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9858a7-dffa-4dca-8648-2e43743884b8 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.245420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9858a7-dffa-4dca-8648-2e43743884b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.245697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9858a7-dffa-4dca-8648-2e43743884b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.253510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07832ac-2097-4670-9d97-954ad02c9b4b None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.256688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9858a7-dffa-4dca-8648-2e43743884b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.258174 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9858a7-dffa-4dca-8648-2e43743884b8 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-7a9858a7-dffa-4dca-8648-2e43743884b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:12.279997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:12.282890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:12.285026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:12.285026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:12.285026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:12.285026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 404/806] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:12.285026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bff497-cba4-4ab1-af11-8800a088a9b1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.285026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bff497-cba4-4ab1-af11-8800a088a9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.285026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bff497-cba4-4ab1-af11-8800a088a9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.299510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82143c79-fc2b-47c4-aba1-8f537272c3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.300562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82143c79-fc2b-47c4-aba1-8f537272c3cb None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.302052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82143c79-fc2b-47c4-aba1-8f537272c3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.302499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82143c79-fc2b-47c4-aba1-8f537272c3cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.303793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82143c79-fc2b-47c4-aba1-8f537272c3cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.306372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82143c79-fc2b-47c4-aba1-8f537272c3cb None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.318333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-43bff497-cba4-4ab1-af11-8800a088a9b1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:12.320772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-43bff497-cba4-4ab1-af11-8800a088a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:12.400228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 405/807] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:12.401005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bff497-cba4-4ab1-af11-8800a088a9b1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.454985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 403/808] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:12.456055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82143c79-fc2b-47c4-aba1-8f537272c3cb None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.518854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:12.548547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bff497-cba4-4ab1-af11-8800a088a9b1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.613205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:12.637909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82143c79-fc2b-47c4-aba1-8f537272c3cb None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.655990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07756269-8554-4ca1-b65a-b6df13f910f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.657352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07756269-8554-4ca1-b65a-b6df13f910f7 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.657640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07756269-8554-4ca1-b65a-b6df13f910f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.658190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07756269-8554-4ca1-b65a-b6df13f910f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.658806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07756269-8554-4ca1-b65a-b6df13f910f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:12.659121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07756269-8554-4ca1-b65a-b6df13f910f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.660544 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07756269-8554-4ca1-b65a-b6df13f910f7 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.681122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4c79-b23c-4fc3-b339-fd7d8e0bbada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.682112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4c79-b23c-4fc3-b339-fd7d8e0bbada None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.682636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4c79-b23c-4fc3-b339-fd7d8e0bbada None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.683135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4c79-b23c-4fc3-b339-fd7d8e0bbada None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.684708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 406/809] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:12.691205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07756269-8554-4ca1-b65a-b6df13f910f7 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.696211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea4c79-b23c-4fc3-b339-fd7d8e0bbada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-e0ea4c79-b23c-4fc3-b339-fd7d8e0bbada None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:12.702435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:12.704129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:12.704990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:12.708308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 404/810] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:12.708308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4c79-b23c-4fc3-b339-fd7d8e0bbada None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.824504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07756269-8554-4ca1-b65a-b6df13f910f7 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.836051 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4c79-b23c-4fc3-b339-fd7d8e0bbada None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:12.867786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d83ff-ea61-4d68-8dcb-c0cea5c5501c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.868453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d83ff-ea61-4d68-8dcb-c0cea5c5501c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.868735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d83ff-ea61-4d68-8dcb-c0cea5c5501c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.869010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d83ff-ea61-4d68-8dcb-c0cea5c5501c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.873283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:12.880749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d83ff-ea61-4d68-8dcb-c0cea5c5501c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.881764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a90c8d-e54b-4b84-8d2d-44c9e5e05840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:12.882216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d83ff-ea61-4d68-8dcb-c0cea5c5501c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.882922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a90c8d-e54b-4b84-8d2d-44c9e5e05840 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:12.883410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a90c8d-e54b-4b84-8d2d-44c9e5e05840 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:12.883897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a90c8d-e54b-4b84-8d2d-44c9e5e05840 None admin] PATH_INFO: `/v3/services/network` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:12.884875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a90c8d-e54b-4b84-8d2d-44c9e5e05840 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:12.887176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a90c8d-e54b-4b84-8d2d-44c9e5e05840 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:12.889787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-aa7d83ff-ea61-4d68-8dcb-c0cea5c5501c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:12.896059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:12.897201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:12.898266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 407/811] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:12.898922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d83ff-ea61-4d68-8dcb-c0cea5c5501c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-b3a90c8d-e54b-4b84-8d2d-44c9e5e05840 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:12.903679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:12.904689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:12.905650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 405/812] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:12.906686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a90c8d-e54b-4b84-8d2d-44c9e5e05840 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.034899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d83ff-ea61-4d68-8dcb-c0cea5c5501c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.049500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a90c8d-e54b-4b84-8d2d-44c9e5e05840 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.077324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949b75f-c817-45c5-b919-0a1863b14951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.077937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949b75f-c817-45c5-b919-0a1863b14951 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.078956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949b75f-c817-45c5-b919-0a1863b14951 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.079263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949b75f-c817-45c5-b919-0a1863b14951 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.080060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949b75f-c817-45c5-b919-0a1863b14951 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:13.080370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949b75f-c817-45c5-b919-0a1863b14951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.082254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949b75f-c817-45c5-b919-0a1863b14951 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.097362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 408/813] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:13.097362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949b75f-c817-45c5-b919-0a1863b14951 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.100718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf7d19-009c-4efb-a71b-ca32196869c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.101593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf7d19-009c-4efb-a71b-ca32196869c3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.102075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf7d19-009c-4efb-a71b-ca32196869c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.102637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf7d19-009c-4efb-a71b-ca32196869c3 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.103588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf7d19-009c-4efb-a71b-ca32196869c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:13.104606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf7d19-009c-4efb-a71b-ca32196869c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.106765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf7d19-009c-4efb-a71b-ca32196869c3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.133692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 406/814] 10.222.0.82 () {56 vars in 1149 bytes} [Wed Aug 24 03:31:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:13.135216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf7d19-009c-4efb-a71b-ca32196869c3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.175087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:13.215448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:13.231364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949b75f-c817-45c5-b919-0a1863b14951 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.263978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf7d19-009c-4efb-a71b-ca32196869c3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.270905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6647611-30cb-402f-a118-9244ca2cdad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.271525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6647611-30cb-402f-a118-9244ca2cdad8 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.271809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6647611-30cb-402f-a118-9244ca2cdad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.272085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6647611-30cb-402f-a118-9244ca2cdad8 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.272728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6647611-30cb-402f-a118-9244ca2cdad8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:13.273036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6647611-30cb-402f-a118-9244ca2cdad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.274557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6647611-30cb-402f-a118-9244ca2cdad8 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.285110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 409/815] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:13.286011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6647611-30cb-402f-a118-9244ca2cdad8 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.306044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5d3b6-a4ba-47d2-a7ec-219abd7d955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.307381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5d3b6-a4ba-47d2-a7ec-219abd7d955f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.307864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5d3b6-a4ba-47d2-a7ec-219abd7d955f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.308511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5d3b6-a4ba-47d2-a7ec-219abd7d955f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.323030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc5d3b6-a4ba-47d2-a7ec-219abd7d955f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.325072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc5d3b6-a4ba-47d2-a7ec-219abd7d955f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-fbc5d3b6-a4ba-47d2-a7ec-219abd7d955f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:13.344442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:13.345691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:13.346633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:13.346633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:13.346633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:13.347738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 407/816] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:13.348544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5d3b6-a4ba-47d2-a7ec-219abd7d955f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.417668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6647611-30cb-402f-a118-9244ca2cdad8 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.450029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:13.462444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2a05d-8fe8-4cdc-84b7-833cd77b04ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.467059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2a05d-8fe8-4cdc-84b7-833cd77b04ae None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.467059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2a05d-8fe8-4cdc-84b7-833cd77b04ae None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.467059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2a05d-8fe8-4cdc-84b7-833cd77b04ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.467313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5d3b6-a4ba-47d2-a7ec-219abd7d955f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.475928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2a05d-8fe8-4cdc-84b7-833cd77b04ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.477407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2a05d-8fe8-4cdc-84b7-833cd77b04ae None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.481403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-abd2a05d-8fe8-4cdc-84b7-833cd77b04ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:13.494797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:13.495828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:13.497524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:13.497524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:13.497524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:13.501442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 410/817] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:13.503097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2a05d-8fe8-4cdc-84b7-833cd77b04ae None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.530682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9e57d-4766-4ba4-941f-8d11818fb0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.534159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9e57d-4766-4ba4-941f-8d11818fb0be None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.535688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9e57d-4766-4ba4-941f-8d11818fb0be None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.535968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9e57d-4766-4ba4-941f-8d11818fb0be None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.536628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9e57d-4766-4ba4-941f-8d11818fb0be None admin] RBAC: Adding query filter params (type=network) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:13.536943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9e57d-4766-4ba4-941f-8d11818fb0be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.538349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9e57d-4766-4ba4-941f-8d11818fb0be None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.546555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 408/818] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:13.548362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9e57d-4766-4ba4-941f-8d11818fb0be None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.656387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2a05d-8fe8-4cdc-84b7-833cd77b04ae None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.676629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9e57d-4766-4ba4-941f-8d11818fb0be None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.695156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f9e7f-8393-46d4-94fb-440691cb510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.696171 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f9e7f-8393-46d4-94fb-440691cb510a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.696654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f9e7f-8393-46d4-94fb-440691cb510a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.697202 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f9e7f-8393-46d4-94fb-440691cb510a None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.698158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f9e7f-8393-46d4-94fb-440691cb510a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:13.698712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f9e7f-8393-46d4-94fb-440691cb510a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.700473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f9e7f-8393-46d4-94fb-440691cb510a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.713867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111d213-340f-4b5f-ad11-d894c8a8fddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.714607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111d213-340f-4b5f-ad11-d894c8a8fddd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.714896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111d213-340f-4b5f-ad11-d894c8a8fddd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.715169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111d213-340f-4b5f-ad11-d894c8a8fddd None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.720272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111d213-340f-4b5f-ad11-d894c8a8fddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:13.720272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111d213-340f-4b5f-ad11-d894c8a8fddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.720272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111d213-340f-4b5f-ad11-d894c8a8fddd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.721098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 411/819] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:13.722191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f9e7f-8393-46d4-94fb-440691cb510a None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.731770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 409/820] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:13.732692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111d213-340f-4b5f-ad11-d894c8a8fddd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.761571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:13.772138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:13.837496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f9e7f-8393-46d4-94fb-440691cb510a None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.843472 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111d213-340f-4b5f-ad11-d894c8a8fddd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:13.875033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54d44c-31ad-4b56-b631-0935886de18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.876582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d44c-31ad-4b56-b631-0935886de18b None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.877136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d44c-31ad-4b56-b631-0935886de18b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.877588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d44c-31ad-4b56-b631-0935886de18b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.878432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54d44c-31ad-4b56-b631-0935886de18b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.879951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54d44c-31ad-4b56-b631-0935886de18b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.880898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20549a3-96cd-4069-940b-310989dba150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:13.881530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20549a3-96cd-4069-940b-310989dba150 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.881815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20549a3-96cd-4069-940b-310989dba150 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.882092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20549a3-96cd-4069-940b-310989dba150 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.889274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20549a3-96cd-4069-940b-310989dba150 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:13.890773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20549a3-96cd-4069-940b-310989dba150 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-c20549a3-96cd-4069-940b-310989dba150 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:13.903619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:13.910527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 410/821] 10.222.0.82 () {56 vars in 1093 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:13.911713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20549a3-96cd-4069-940b-310989dba150 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:13.966778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 412/822] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:13.968015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c258-c023-42e4-b3a1-a368ce9a50b8 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:13.968311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c258-c023-42e4-b3a1-a368ce9a50b8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:13.968589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c258-c023-42e4-b3a1-a368ce9a50b8 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:13.975349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 413/823] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:13.976333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57c258-c023-42e4-b3a1-a368ce9a50b8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.024945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:14.037938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20549a3-96cd-4069-940b-310989dba150 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.090788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a4589-dc5f-4a41-a978-9695897ebdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.091687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a4589-dc5f-4a41-a978-9695897ebdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.091687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a4589-dc5f-4a41-a978-9695897ebdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.091973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a4589-dc5f-4a41-a978-9695897ebdf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.101105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a4589-dc5f-4a41-a978-9695897ebdf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.102648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a4589-dc5f-4a41-a978-9695897ebdf1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.113778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-985a4589-dc5f-4a41-a978-9695897ebdf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:14.115724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:14.116625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:14.117526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:14.117526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:14.117526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:14.117526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:14.117526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:14.117526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:14.117526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:14.117526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:14.120720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57c258-c023-42e4-b3a1-a368ce9a50b8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.121845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 411/824] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:14.123751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b928ed8-6771-49a8-80d5-0dcb242d7d1e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.123751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b928ed8-6771-49a8-80d5-0dcb242d7d1e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.123932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b928ed8-6771-49a8-80d5-0dcb242d7d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.143036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-8b928ed8-6771-49a8-80d5-0dcb242d7d1e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:14.144423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8b928ed8-6771-49a8-80d5-0dcb242d7d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:14.163733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77699ea0-0bc6-4366-90f1-0a41ba5cf5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.164522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77699ea0-0bc6-4366-90f1-0a41ba5cf5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.164795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77699ea0-0bc6-4366-90f1-0a41ba5cf5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.165072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77699ea0-0bc6-4366-90f1-0a41ba5cf5a7 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.165821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77699ea0-0bc6-4366-90f1-0a41ba5cf5a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:14.166123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77699ea0-0bc6-4366-90f1-0a41ba5cf5a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.167609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77699ea0-0bc6-4366-90f1-0a41ba5cf5a7 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.192397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 414/825] 10.222.0.82 () {56 vars in 1133 bytes} [Wed Aug 24 03:31:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:14.193472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77699ea0-0bc6-4366-90f1-0a41ba5cf5a7 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.248090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 412/826] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:14.248090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b928ed8-6771-49a8-80d5-0dcb242d7d1e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.315375 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77699ea0-0bc6-4366-90f1-0a41ba5cf5a7 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.332905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:14.352940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941cb5-4a40-4667-85e0-254b678d9a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.353843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941cb5-4a40-4667-85e0-254b678d9a7c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.354325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941cb5-4a40-4667-85e0-254b678d9a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.354790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941cb5-4a40-4667-85e0-254b678d9a7c None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.355664 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba941cb5-4a40-4667-85e0-254b678d9a7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:14.356145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba941cb5-4a40-4667-85e0-254b678d9a7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.356345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b928ed8-6771-49a8-80d5-0dcb242d7d1e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.357941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba941cb5-4a40-4667-85e0-254b678d9a7c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.371333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 415/827] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:14.376812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941cb5-4a40-4667-85e0-254b678d9a7c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.391933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:14.410166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04693d30-0198-45ad-81aa-f55c3e4cb6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.411055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04693d30-0198-45ad-81aa-f55c3e4cb6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.411517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04693d30-0198-45ad-81aa-f55c3e4cb6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.411943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04693d30-0198-45ad-81aa-f55c3e4cb6d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.488043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04693d30-0198-45ad-81aa-f55c3e4cb6d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.489773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04693d30-0198-45ad-81aa-f55c3e4cb6d6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.507010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941cb5-4a40-4667-85e0-254b678d9a7c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.554947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58977d2c-fc2d-4904-b6a2-3ef9e8560981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.555641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58977d2c-fc2d-4904-b6a2-3ef9e8560981 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.555920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58977d2c-fc2d-4904-b6a2-3ef9e8560981 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.556199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58977d2c-fc2d-4904-b6a2-3ef9e8560981 None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/b14c2eb2840b46538f169c39f200639f/roles/9d16c3fb52ac40068ddddbf66411c519` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.572802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58977d2c-fc2d-4904-b6a2-3ef9e8560981 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=b14c2eb2840b46538f169c39f200639f, role_id=9d16c3fb52ac40068ddddbf66411c519)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.574287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58977d2c-fc2d-4904-b6a2-3ef9e8560981 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.581967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:14.604541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 413/828] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:31:14.605486 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04693d30-0198-45ad-81aa-f55c3e4cb6d6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.662855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 416/829] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:31:14 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/b14c2eb2840b46538f169c39f200639f/roles/9d16c3fb52ac40068ddddbf66411c519 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:31:14.664007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58977d2c-fc2d-4904-b6a2-3ef9e8560981 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.716077 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04693d30-0198-45ad-81aa-f55c3e4cb6d6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.729508 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:14.762835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c6a63-78b0-474b-b489-df16619db6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.763270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c6a63-78b0-474b-b489-df16619db6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.763530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c6a63-78b0-474b-b489-df16619db6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.763802 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c6a63-78b0-474b-b489-df16619db6a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.776343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c6a63-78b0-474b-b489-df16619db6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:14.780591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58977d2c-fc2d-4904-b6a2-3ef9e8560981 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-d88c6a63-78b0-474b-b489-df16619db6a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:14.781664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:14.783212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:14.784521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 414/830] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:14.785913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6af83c-4566-43c1-bfc8-c451c814c379 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.785913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6af83c-4566-43c1-bfc8-c451c814c379 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.785913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6af83c-4566-43c1-bfc8-c451c814c379 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.786916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 415/831] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:14.791847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6af83c-4566-43c1-bfc8-c451c814c379 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.827894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb00b3c-d2b7-4f70-bd01-e76a0b3d0331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.829112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb00b3c-d2b7-4f70-bd01-e76a0b3d0331 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.829423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb00b3c-d2b7-4f70-bd01-e76a0b3d0331 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.829704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb00b3c-d2b7-4f70-bd01-e76a0b3d0331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.841433 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb00b3c-d2b7-4f70-bd01-e76a0b3d0331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.842933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb00b3c-d2b7-4f70-bd01-e76a0b3d0331 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.843851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-aeb00b3c-d2b7-4f70-bd01-e76a0b3d0331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:14.858548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:14.860594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:14.861700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:14.861700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:14.861700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:14.861700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 417/832] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:14.861700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d1193-8290-4aac-8e85-5e259ba0d0af None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.861700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d1193-8290-4aac-8e85-5e259ba0d0af None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.861700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d1193-8290-4aac-8e85-5e259ba0d0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.881196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-c02d1193-8290-4aac-8e85-5e259ba0d0af None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:14.882416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c02d1193-8290-4aac-8e85-5e259ba0d0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:14.919297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6af83c-4566-43c1-bfc8-c451c814c379 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:14.957238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b836f9-42d9-40ea-841b-d5fed1500d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:14.957764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b836f9-42d9-40ea-841b-d5fed1500d54 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:14.958049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b836f9-42d9-40ea-841b-d5fed1500d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:14.958355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b836f9-42d9-40ea-841b-d5fed1500d54 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:14.959035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b836f9-42d9-40ea-841b-d5fed1500d54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:14.959411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b836f9-42d9-40ea-841b-d5fed1500d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:14.960774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b836f9-42d9-40ea-841b-d5fed1500d54 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:14.962610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 418/833] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:14.962610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d1193-8290-4aac-8e85-5e259ba0d0af None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:14.983703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 416/834] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:14.985629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b836f9-42d9-40ea-841b-d5fed1500d54 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.029650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.104226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d1193-8290-4aac-8e85-5e259ba0d0af None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.118851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.125921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b836f9-42d9-40ea-841b-d5fed1500d54 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.143853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b1b69-c7a3-4345-979d-66af2a5c3e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.144419 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b1b69-c7a3-4345-979d-66af2a5c3e7f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.144796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b1b69-c7a3-4345-979d-66af2a5c3e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.145004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b1b69-c7a3-4345-979d-66af2a5c3e7f None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.145663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b1b69-c7a3-4345-979d-66af2a5c3e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:15.146025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b1b69-c7a3-4345-979d-66af2a5c3e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.147631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b1b69-c7a3-4345-979d-66af2a5c3e7f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.162131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 419/835] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:15.167828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b1b69-c7a3-4345-979d-66af2a5c3e7f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.170103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812a94-048e-4cfe-ada9-e3f307605f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.171010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812a94-048e-4cfe-ada9-e3f307605f89 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.171480 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812a94-048e-4cfe-ada9-e3f307605f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.171961 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812a94-048e-4cfe-ada9-e3f307605f89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.185010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07812a94-048e-4cfe-ada9-e3f307605f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.186808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07812a94-048e-4cfe-ada9-e3f307605f89 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-07812a94-048e-4cfe-ada9-e3f307605f89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:15.205893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:15.206972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:15.207881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:15.207881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:15.207881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:15.210010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 417/836] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:15.218244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812a94-048e-4cfe-ada9-e3f307605f89 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.278274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.282858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b1b69-c7a3-4345-979d-66af2a5c3e7f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.318077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ed1d8-b3a7-4cd8-bc3f-afd73a0cedb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.321112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ed1d8-b3a7-4cd8-bc3f-afd73a0cedb9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.321112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ed1d8-b3a7-4cd8-bc3f-afd73a0cedb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.321112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ed1d8-b3a7-4cd8-bc3f-afd73a0cedb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.327727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ed1d8-b3a7-4cd8-bc3f-afd73a0cedb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.329730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ed1d8-b3a7-4cd8-bc3f-afd73a0cedb9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.332299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812a94-048e-4cfe-ada9-e3f307605f89 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-100ed1d8-b3a7-4cd8-bc3f-afd73a0cedb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:15.341495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:15.342652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:15.343776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:15.343776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:15.343776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:15.343776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:15.343776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:15.343776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:15.343776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:15.343776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:15.349645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 420/837] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:15.350329 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ed1d8-b3a7-4cd8-bc3f-afd73a0cedb9 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.373585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eab1b3-729a-4073-b9be-ec49cfa85244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.374199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eab1b3-729a-4073-b9be-ec49cfa85244 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.374488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eab1b3-729a-4073-b9be-ec49cfa85244 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.374796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eab1b3-729a-4073-b9be-ec49cfa85244 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.376036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eab1b3-729a-4073-b9be-ec49cfa85244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:15.376358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eab1b3-729a-4073-b9be-ec49cfa85244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.377750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eab1b3-729a-4073-b9be-ec49cfa85244 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.389508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 418/838] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:15.390303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eab1b3-729a-4073-b9be-ec49cfa85244 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.407480 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.475794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ed1d8-b3a7-4cd8-bc3f-afd73a0cedb9 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.506311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eab1b3-729a-4073-b9be-ec49cfa85244 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.512087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83061300-d924-419f-a00a-7d905a0e884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.512731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061300-d924-419f-a00a-7d905a0e884f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.513015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061300-d924-419f-a00a-7d905a0e884f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.513293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061300-d924-419f-a00a-7d905a0e884f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.513890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83061300-d924-419f-a00a-7d905a0e884f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.515299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83061300-d924-419f-a00a-7d905a0e884f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-83061300-d924-419f-a00a-7d905a0e884f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:15.525162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:15.526176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:15.527066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 421/839] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:15.527898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83061300-d924-419f-a00a-7d905a0e884f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.547897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232c5ad-7edb-4030-9421-8db12c250ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.548840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232c5ad-7edb-4030-9421-8db12c250ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.549398 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232c5ad-7edb-4030-9421-8db12c250ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.549896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232c5ad-7edb-4030-9421-8db12c250ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.550837 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232c5ad-7edb-4030-9421-8db12c250ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:15.558867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232c5ad-7edb-4030-9421-8db12c250ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.560531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232c5ad-7edb-4030-9421-8db12c250ee1 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.570360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.581345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 419/840] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:15.593521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232c5ad-7edb-4030-9421-8db12c250ee1 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.654801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83061300-d924-419f-a00a-7d905a0e884f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.668678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.693272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee93b8d-8a24-4839-8f92-a138722df4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.693909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee93b8d-8a24-4839-8f92-a138722df4ed None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.694185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee93b8d-8a24-4839-8f92-a138722df4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.694463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee93b8d-8a24-4839-8f92-a138722df4ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.703001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee93b8d-8a24-4839-8f92-a138722df4ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.703173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232c5ad-7edb-4030-9421-8db12c250ee1 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.705565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee93b8d-8a24-4839-8f92-a138722df4ed None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-0ee93b8d-8a24-4839-8f92-a138722df4ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:15.717716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:15.718754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:15.719772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 422/841] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:15.720394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee93b8d-8a24-4839-8f92-a138722df4ed None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.746039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0997a-a5cf-4e00-a265-54a29bb6f2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.746684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0997a-a5cf-4e00-a265-54a29bb6f2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.746970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0997a-a5cf-4e00-a265-54a29bb6f2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.747245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0997a-a5cf-4e00-a265-54a29bb6f2a9 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.747884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0997a-a5cf-4e00-a265-54a29bb6f2a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:15.748187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0997a-a5cf-4e00-a265-54a29bb6f2a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.749556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e0997a-a5cf-4e00-a265-54a29bb6f2a9 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.755032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 420/842] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:15.755658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0997a-a5cf-4e00-a265-54a29bb6f2a9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.819477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.836131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee93b8d-8a24-4839-8f92-a138722df4ed None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.866147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0997a-a5cf-4e00-a265-54a29bb6f2a9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:15.878048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab68e5-7404-46e0-8d73-d3d898ab0299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.879580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab68e5-7404-46e0-8d73-d3d898ab0299 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.880144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab68e5-7404-46e0-8d73-d3d898ab0299 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.880629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab68e5-7404-46e0-8d73-d3d898ab0299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.881738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab68e5-7404-46e0-8d73-d3d898ab0299 None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=0f34fe3fe912498b90dbf950e7fc999d) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:15.882226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab68e5-7404-46e0-8d73-d3d898ab0299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.884411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab68e5-7404-46e0-8d73-d3d898ab0299 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.900048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 423/843] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/role_assignments?user.id=0f34fe3fe912498b90dbf950e7fc999d&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9e904216086e4a06abf9216b0c863f89 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:15.900048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab68e5-7404-46e0-8d73-d3d898ab0299 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:15.926713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf52a5-1511-4f94-9ea1-037e2fa0d64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:15.927786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf52a5-1511-4f94-9ea1-037e2fa0d64b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:15.928273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf52a5-1511-4f94-9ea1-037e2fa0d64b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:15.928765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf52a5-1511-4f94-9ea1-037e2fa0d64b None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:15.930986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf52a5-1511-4f94-9ea1-037e2fa0d64b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:15.931308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf52a5-1511-4f94-9ea1-037e2fa0d64b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:15.932824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf52a5-1511-4f94-9ea1-037e2fa0d64b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:15.938532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:15.967049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 421/844] 10.222.0.82 () {56 vars in 1135 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:15.972780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf52a5-1511-4f94-9ea1-037e2fa0d64b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:16.042958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab68e5-7404-46e0-8d73-d3d898ab0299 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:16.084020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf52a5-1511-4f94-9ea1-037e2fa0d64b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:16.086583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410d507-7ce1-4bb8-bf82-2e1508c54ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:16.087474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410d507-7ce1-4bb8-bf82-2e1508c54ffe None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.087987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410d507-7ce1-4bb8-bf82-2e1508c54ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.088459 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410d507-7ce1-4bb8-bf82-2e1508c54ffe None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.089112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410d507-7ce1-4bb8-bf82-2e1508c54ffe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:16.089426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410d507-7ce1-4bb8-bf82-2e1508c54ffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:16.090832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410d507-7ce1-4bb8-bf82-2e1508c54ffe None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:16.096350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 424/845] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:16.128631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee74f5b-bc23-4fc1-a82b-e2cd55ed3f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:16.128631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee74f5b-bc23-4fc1-a82b-e2cd55ed3f62 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.128631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee74f5b-bc23-4fc1-a82b-e2cd55ed3f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.128631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee74f5b-bc23-4fc1-a82b-e2cd55ed3f62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.203587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:16.212352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee74f5b-bc23-4fc1-a82b-e2cd55ed3f62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:16.213852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee74f5b-bc23-4fc1-a82b-e2cd55ed3f62 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:16.335635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 422/846] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:31:16.341307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410d507-7ce1-4bb8-bf82-2e1508c54ffe None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:16.359162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:16.452736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410d507-7ce1-4bb8-bf82-2e1508c54ffe None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:16.491288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388df8d-cb63-4d11-b1f2-ab1e84dcf7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:16.491946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388df8d-cb63-4d11-b1f2-ab1e84dcf7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.492231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388df8d-cb63-4d11-b1f2-ab1e84dcf7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.492506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388df8d-cb63-4d11-b1f2-ab1e84dcf7d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.504055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388df8d-cb63-4d11-b1f2-ab1e84dcf7d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:16.505586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388df8d-cb63-4d11-b1f2-ab1e84dcf7d6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-f388df8d-cb63-4d11-b1f2-ab1e84dcf7d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:16.521476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:16.522520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:16.523436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:16.523436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:16.523436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:16.523436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 425/847] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:16.532231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee74f5b-bc23-4fc1-a82b-e2cd55ed3f62 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:16.637811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c446bb-aa21-47b3-8d72-1a3f4a6d7ac3 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.638111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c446bb-aa21-47b3-8d72-1a3f4a6d7ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.638353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c446bb-aa21-47b3-8d72-1a3f4a6d7ac3 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.639930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 426/848] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:16.640552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:16.645790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b8c4f-2bdf-48e6-8435-87d7cde18dd9 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.646078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b8c4f-2bdf-48e6-8435-87d7cde18dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.646361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b8c4f-2bdf-48e6-8435-87d7cde18dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.648447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee74f5b-bc23-4fc1-a82b-e2cd55ed3f62 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:16.665135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-4e4b8c4f-2bdf-48e6-8435-87d7cde18dd9 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:16.666337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4e4b8c4f-2bdf-48e6-8435-87d7cde18dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:16.685978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f42cc-7956-424f-b0fc-30d8cf39c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:16.686623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f42cc-7956-424f-b0fc-30d8cf39c1db None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.686907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f42cc-7956-424f-b0fc-30d8cf39c1db None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.687179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f42cc-7956-424f-b0fc-30d8cf39c1db None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.688438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f42cc-7956-424f-b0fc-30d8cf39c1db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:16.688985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f42cc-7956-424f-b0fc-30d8cf39c1db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:16.690488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f42cc-7956-424f-b0fc-30d8cf39c1db None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:16.702982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 423/849] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:16.711655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f42cc-7956-424f-b0fc-30d8cf39c1db None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:16.731910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:16.745582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 427/850] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:16.826264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f42cc-7956-424f-b0fc-30d8cf39c1db None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:16.863600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697741c-755d-4656-b9d2-a1b7164c1c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:16.864506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697741c-755d-4656-b9d2-a1b7164c1c19 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:16.864938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697741c-755d-4656-b9d2-a1b7164c1c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:16.865360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697741c-755d-4656-b9d2-a1b7164c1c19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:16.872437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697741c-755d-4656-b9d2-a1b7164c1c19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:16.874049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697741c-755d-4656-b9d2-a1b7164c1c19 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-7697741c-755d-4656-b9d2-a1b7164c1c19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:16.886240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:16.887357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:16.888563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:16.888563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:16.888563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:16.888563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:16.888563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:16.888563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:16.888563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:16.888563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:16.890588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 424/851] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:16.899653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b8c4f-2bdf-48e6-8435-87d7cde18dd9 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.011647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b8c4f-2bdf-48e6-8435-87d7cde18dd9 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.047795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697741c-755d-4656-b9d2-a1b7164c1c19 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.050808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87efd7-7354-469d-a38a-9b8a3f3dc4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.051749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87efd7-7354-469d-a38a-9b8a3f3dc4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.052034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87efd7-7354-469d-a38a-9b8a3f3dc4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.052318 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87efd7-7354-469d-a38a-9b8a3f3dc4b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.054013 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87efd7-7354-469d-a38a-9b8a3f3dc4b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:17.054343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87efd7-7354-469d-a38a-9b8a3f3dc4b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.055837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87efd7-7354-469d-a38a-9b8a3f3dc4b7 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.068007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:17.078641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 428/852] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:17.080112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87efd7-7354-469d-a38a-9b8a3f3dc4b7 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.142973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:17.174322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697741c-755d-4656-b9d2-a1b7164c1c19 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.201501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87efd7-7354-469d-a38a-9b8a3f3dc4b7 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.213259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64b33-1c89-4826-aebe-1a010c2312e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.214240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64b33-1c89-4826-aebe-1a010c2312e9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.214776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64b33-1c89-4826-aebe-1a010c2312e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.215254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64b33-1c89-4826-aebe-1a010c2312e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.228849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c64b33-1c89-4826-aebe-1a010c2312e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-02c64b33-1c89-4826-aebe-1a010c2312e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:17.235352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:17.236549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:31:17.237434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:17.245187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 425/853] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:31:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:17.246631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64b33-1c89-4826-aebe-1a010c2312e9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.248880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fd65e-e751-4fb4-8d65-970e25ead8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.249616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fd65e-e751-4fb4-8d65-970e25ead8aa None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.250363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fd65e-e751-4fb4-8d65-970e25ead8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.250577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fd65e-e751-4fb4-8d65-970e25ead8aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.251557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fd65e-e751-4fb4-8d65-970e25ead8aa None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=ad67f6ace29a4591bf2b8b4f119b3323) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:17.252822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fd65e-e751-4fb4-8d65-970e25ead8aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.253643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fd65e-e751-4fb4-8d65-970e25ead8aa None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.266560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 429/854] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:17 2022] GET /identity/v3/role_assignments?user.id=ad67f6ace29a4591bf2b8b4f119b3323&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9e904216086e4a06abf9216b0c863f89 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:17.379600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:17.411499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64b33-1c89-4826-aebe-1a010c2312e9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.455623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e26e8-8c98-4efd-af42-25d83651bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.456308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e26e8-8c98-4efd-af42-25d83651bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.456594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e26e8-8c98-4efd-af42-25d83651bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.456870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e26e8-8c98-4efd-af42-25d83651bdf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.457566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e26e8-8c98-4efd-af42-25d83651bdf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:17.457862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e26e8-8c98-4efd-af42-25d83651bdf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.459310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e26e8-8c98-4efd-af42-25d83651bdf9 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.469151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 426/855] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:31:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:17.475149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fd65e-e751-4fb4-8d65-970e25ead8aa None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.596555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fd65e-e751-4fb4-8d65-970e25ead8aa None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.616081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:17.642325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230ddab-def6-47f5-b321-93ef1c5baa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.643176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230ddab-def6-47f5-b321-93ef1c5baa62 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.643465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230ddab-def6-47f5-b321-93ef1c5baa62 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.643745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230ddab-def6-47f5-b321-93ef1c5baa62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.660264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230ddab-def6-47f5-b321-93ef1c5baa62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.661798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230ddab-def6-47f5-b321-93ef1c5baa62 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-2230ddab-def6-47f5-b321-93ef1c5baa62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:17.678109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:17.679426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:17.680854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:17.680854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:17.680854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:17.680854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 430/856] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:17.685347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e26e8-8c98-4efd-af42-25d83651bdf9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.798593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e26e8-8c98-4efd-af42-25d83651bdf9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:17.834413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7f8ad-8bcf-4fdf-adc4-e6ca9583bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:17.835293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7f8ad-8bcf-4fdf-adc4-e6ca9583bc3a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:17.835746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7f8ad-8bcf-4fdf-adc4-e6ca9583bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:17.836164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7f8ad-8bcf-4fdf-adc4-e6ca9583bc3a None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:17.836949 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7f8ad-8bcf-4fdf-adc4-e6ca9583bc3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:17.837395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7f8ad-8bcf-4fdf-adc4-e6ca9583bc3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:17.838941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7f8ad-8bcf-4fdf-adc4-e6ca9583bc3a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:17.841652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:17.854028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 427/857] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:17.859499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230ddab-def6-47f5-b321-93ef1c5baa62 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:17.983935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230ddab-def6-47f5-b321-93ef1c5baa62 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.023802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63891ca7-b05c-4f40-9dc1-8aadc74356e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.024515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63891ca7-b05c-4f40-9dc1-8aadc74356e0 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.024806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63891ca7-b05c-4f40-9dc1-8aadc74356e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.025086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63891ca7-b05c-4f40-9dc1-8aadc74356e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.035113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63891ca7-b05c-4f40-9dc1-8aadc74356e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.036589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63891ca7-b05c-4f40-9dc1-8aadc74356e0 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-63891ca7-b05c-4f40-9dc1-8aadc74356e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:18.049576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:18.050561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:18.051938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 431/858] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:31:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:18.057002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7f8ad-8bcf-4fdf-adc4-e6ca9583bc3a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.129347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9bea8-a1dc-4995-81b7-b2c05464ad27 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.129693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9bea8-a1dc-4995-81b7-b2c05464ad27 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.129906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9bea8-a1dc-4995-81b7-b2c05464ad27 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.131670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 432/859] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:18.136748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622eaef-658b-49dd-a004-c048aad7e348 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.137036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622eaef-658b-49dd-a004-c048aad7e348 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.137315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622eaef-658b-49dd-a004-c048aad7e348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.156482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-c622eaef-658b-49dd-a004-c048aad7e348 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:18.157851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c622eaef-658b-49dd-a004-c048aad7e348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.181783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7f8ad-8bcf-4fdf-adc4-e6ca9583bc3a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.193120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:18.223341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05c6cf-c475-49cd-9be0-f57a050d8c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.224346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05c6cf-c475-49cd-9be0-f57a050d8c42 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.224632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05c6cf-c475-49cd-9be0-f57a050d8c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.225191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05c6cf-c475-49cd-9be0-f57a050d8c42 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.225940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05c6cf-c475-49cd-9be0-f57a050d8c42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:18.226413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05c6cf-c475-49cd-9be0-f57a050d8c42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.228184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05c6cf-c475-49cd-9be0-f57a050d8c42 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.246770 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 433/860] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:18.248746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce18a-6269-4786-a8bf-0ac069078fe9 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.249240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce18a-6269-4786-a8bf-0ac069078fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.249732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce18a-6269-4786-a8bf-0ac069078fe9 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.255756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 428/861] 10.222.0.82 () {56 vars in 1149 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:18.256444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 434/862] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:18.257276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05c6cf-c475-49cd-9be0-f57a050d8c42 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.257973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf039ca-345d-46a5-a79a-f1c91d414d95 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.258483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf039ca-345d-46a5-a79a-f1c91d414d95 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.258998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf039ca-345d-46a5-a79a-f1c91d414d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.280868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-fbf039ca-345d-46a5-a79a-f1c91d414d95 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:18.282650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fbf039ca-345d-46a5-a79a-f1c91d414d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:18.326004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:18.374530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 435/863] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:18.389291 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05c6cf-c475-49cd-9be0-f57a050d8c42 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.427280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e246ce-3edd-4b40-9cd9-f7c24cf150e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.427951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e246ce-3edd-4b40-9cd9-f7c24cf150e0 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.428298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e246ce-3edd-4b40-9cd9-f7c24cf150e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.428580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e246ce-3edd-4b40-9cd9-f7c24cf150e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.504229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e246ce-3edd-4b40-9cd9-f7c24cf150e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.505625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e246ce-3edd-4b40-9cd9-f7c24cf150e0 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.525771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf039ca-345d-46a5-a79a-f1c91d414d95 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.575608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:18.639479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 429/864] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:31:18.650970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e246ce-3edd-4b40-9cd9-f7c24cf150e0 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.669279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:18.679994 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf039ca-345d-46a5-a79a-f1c91d414d95 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.731326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13c239-cd3b-4c5b-b908-6ed639072b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.732482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13c239-cd3b-4c5b-b908-6ed639072b27 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.732992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13c239-cd3b-4c5b-b908-6ed639072b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.734218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13c239-cd3b-4c5b-b908-6ed639072b27 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.735114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13c239-cd3b-4c5b-b908-6ed639072b27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.736682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13c239-cd3b-4c5b-b908-6ed639072b27 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-cd13c239-cd3b-4c5b-b908-6ed639072b27 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:18.748924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:18.749927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:31:18.752028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:18.752822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 436/865] 10.222.0.82 () {56 vars in 1132 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:18.752900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13c239-cd3b-4c5b-b908-6ed639072b27 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.769851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e246ce-3edd-4b40-9cd9-f7c24cf150e0 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.808786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4639b-fba4-4c32-83b3-66c57e9ae8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.809425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4639b-fba4-4c32-83b3-66c57e9ae8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.809714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4639b-fba4-4c32-83b3-66c57e9ae8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.809992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4639b-fba4-4c32-83b3-66c57e9ae8a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.824053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4639b-fba4-4c32-83b3-66c57e9ae8a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.825750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4639b-fba4-4c32-83b3-66c57e9ae8a7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-94d4639b-fba4-4c32-83b3-66c57e9ae8a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:18.845025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:18.846072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:18.847040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:18.847040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:18.847040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:18.850898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 430/866] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:18.851923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4639b-fba4-4c32-83b3-66c57e9ae8a7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:18.863294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:18.877843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13c239-cd3b-4c5b-b908-6ed639072b27 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:18.932125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30112271-d232-4411-a990-e7cede418f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:18.932742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30112271-d232-4411-a990-e7cede418f41 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:18.933029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30112271-d232-4411-a990-e7cede418f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:18.933303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30112271-d232-4411-a990-e7cede418f41 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:18.933953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30112271-d232-4411-a990-e7cede418f41 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:18.935376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30112271-d232-4411-a990-e7cede418f41 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:18.986833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4639b-fba4-4c32-83b3-66c57e9ae8a7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.018678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 437/867] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:31:18 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:19.018678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30112271-d232-4411-a990-e7cede418f41 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.043816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:19.046817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8ba3-794d-4ba1-a628-03d91d22f2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.047508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8ba3-794d-4ba1-a628-03d91d22f2dd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.047786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8ba3-794d-4ba1-a628-03d91d22f2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.048061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8ba3-794d-4ba1-a628-03d91d22f2dd None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.048716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8ba3-794d-4ba1-a628-03d91d22f2dd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:19.049069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8ba3-794d-4ba1-a628-03d91d22f2dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.051204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8ba3-794d-4ba1-a628-03d91d22f2dd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.058676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 431/868] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:19.063712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8ba3-794d-4ba1-a628-03d91d22f2dd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.152285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:19.174439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30112271-d232-4411-a990-e7cede418f41 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.195680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8ba3-794d-4ba1-a628-03d91d22f2dd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.213410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef1b11-4348-458e-bcf7-c2d442c5f6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.214029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef1b11-4348-458e-bcf7-c2d442c5f6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.214315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef1b11-4348-458e-bcf7-c2d442c5f6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.215266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef1b11-4348-458e-bcf7-c2d442c5f6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.215266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef1b11-4348-458e-bcf7-c2d442c5f6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:19.215573 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef1b11-4348-458e-bcf7-c2d442c5f6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.217069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef1b11-4348-458e-bcf7-c2d442c5f6c2 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.231584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 438/869] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:19.236737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef1b11-4348-458e-bcf7-c2d442c5f6c2 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.241621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686b97c-9a41-4474-8cb2-863e26057160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.242582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686b97c-9a41-4474-8cb2-863e26057160 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.243074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686b97c-9a41-4474-8cb2-863e26057160 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.243553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686b97c-9a41-4474-8cb2-863e26057160 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.244358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686b97c-9a41-4474-8cb2-863e26057160 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:19.244671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686b97c-9a41-4474-8cb2-863e26057160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.246063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686b97c-9a41-4474-8cb2-863e26057160 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.252141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 432/870] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:19.336520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:19.366611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef1b11-4348-458e-bcf7-c2d442c5f6c2 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.407816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e97f5-d193-4ec5-abb8-63979da710b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.408476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e97f5-d193-4ec5-abb8-63979da710b5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.408763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e97f5-d193-4ec5-abb8-63979da710b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.409043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e97f5-d193-4ec5-abb8-63979da710b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.417148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e97f5-d193-4ec5-abb8-63979da710b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.418668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e97f5-d193-4ec5-abb8-63979da710b5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-273e97f5-d193-4ec5-abb8-63979da710b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:19.431989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:19.432949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:19.433865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:19.433865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:19.433865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:19.433865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:19.433865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:19.433865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:19.433865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:19.433865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:19.433865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 439/871] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:19.441819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686b97c-9a41-4474-8cb2-863e26057160 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.499502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bd761-ab7d-46b6-9441-82d0b30abb37 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.500688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bd761-ab7d-46b6-9441-82d0b30abb37 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.501409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bd761-ab7d-46b6-9441-82d0b30abb37 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.503333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 440/872] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:19.511041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712322f-839d-430b-afcf-1fc7cab08e3c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.511607 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712322f-839d-430b-afcf-1fc7cab08e3c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.512206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712322f-839d-430b-afcf-1fc7cab08e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.532903 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-2712322f-839d-430b-afcf-1fc7cab08e3c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:19.534924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2712322f-839d-430b-afcf-1fc7cab08e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:19.567866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686b97c-9a41-4474-8cb2-863e26057160 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.619366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:19.626959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18705c65-4397-4764-9d13-8bb93b35e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.627594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18705c65-4397-4764-9d13-8bb93b35e191 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.627869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18705c65-4397-4764-9d13-8bb93b35e191 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.628148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18705c65-4397-4764-9d13-8bb93b35e191 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.631988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18705c65-4397-4764-9d13-8bb93b35e191 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:19.632090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 441/873] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:19.632171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18705c65-4397-4764-9d13-8bb93b35e191 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.632171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18705c65-4397-4764-9d13-8bb93b35e191 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.644686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 433/874] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:19.650627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712322f-839d-430b-afcf-1fc7cab08e3c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:19.714309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:19.759422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712322f-839d-430b-afcf-1fc7cab08e3c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:19.799019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30779a62-6638-4423-a184-0bea6dd4f26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:19.799714 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779a62-6638-4423-a184-0bea6dd4f26f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:19.800004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779a62-6638-4423-a184-0bea6dd4f26f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:19.800283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779a62-6638-4423-a184-0bea6dd4f26f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:19.801022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30779a62-6638-4423-a184-0bea6dd4f26f None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=b14c2eb2840b46538f169c39f200639f) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:19.801337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30779a62-6638-4423-a184-0bea6dd4f26f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:19.802813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30779a62-6638-4423-a184-0bea6dd4f26f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:19.809027 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 442/875] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/role_assignments?user.id=b14c2eb2840b46538f169c39f200639f&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:19.911282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18705c65-4397-4764-9d13-8bb93b35e191 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.027503 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18705c65-4397-4764-9d13-8bb93b35e191 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:20.065180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9f9a0-37c3-416d-8122-90410eb245ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:20.065821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9f9a0-37c3-416d-8122-90410eb245ce None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.066107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9f9a0-37c3-416d-8122-90410eb245ce None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.066380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9f9a0-37c3-416d-8122-90410eb245ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.067015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9f9a0-37c3-416d-8122-90410eb245ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:20.068398 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9f9a0-37c3-416d-8122-90410eb245ce None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-5df9f9a0-37c3-416d-8122-90410eb245ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:20.077660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:20.078781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:20.079829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 434/876] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:20.086798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30779a62-6638-4423-a184-0bea6dd4f26f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.202355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30779a62-6638-4423-a184-0bea6dd4f26f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:20.236282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:20.244491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60355674-5fd4-4118-9cb3-9cb8bab9c279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:20.245108 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60355674-5fd4-4118-9cb3-9cb8bab9c279 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.245394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60355674-5fd4-4118-9cb3-9cb8bab9c279 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.245673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60355674-5fd4-4118-9cb3-9cb8bab9c279 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.246309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60355674-5fd4-4118-9cb3-9cb8bab9c279 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:20.246647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60355674-5fd4-4118-9cb3-9cb8bab9c279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:20.248035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60355674-5fd4-4118-9cb3-9cb8bab9c279 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:20.254237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 443/877] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:20.260029 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9f9a0-37c3-416d-8122-90410eb245ce None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:20.323549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:20.372311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9f9a0-37c3-416d-8122-90410eb245ce None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:20.418716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d0bad-bd27-4239-8e51-3098095f9c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:20.419681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0bad-bd27-4239-8e51-3098095f9c93 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:20.420188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0bad-bd27-4239-8e51-3098095f9c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:20.420662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0bad-bd27-4239-8e51-3098095f9c93 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:20.421488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d0bad-bd27-4239-8e51-3098095f9c93 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:20.422006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d0bad-bd27-4239-8e51-3098095f9c93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:20.424066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d0bad-bd27-4239-8e51-3098095f9c93 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:20.429322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 435/878] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:21.167157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18f525-a7f7-4625-ab15-72521366dba9 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.167925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18f525-a7f7-4625-ab15-72521366dba9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.167925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18f525-a7f7-4625-ab15-72521366dba9 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.169339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 444/879] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:21.176255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2091b6-c55c-485e-9b1f-0e775c498f13 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.176792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2091b6-c55c-485e-9b1f-0e775c498f13 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.176903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2091b6-c55c-485e-9b1f-0e775c498f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.201522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-ee2091b6-c55c-485e-9b1f-0e775c498f13 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:21.202813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ee2091b6-c55c-485e-9b1f-0e775c498f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:21.282251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 436/880] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:21.315639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82ca76-22a0-4437-b2e1-2d0e69fa72de None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.315959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82ca76-22a0-4437-b2e1-2d0e69fa72de None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.316210 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82ca76-22a0-4437-b2e1-2d0e69fa72de None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.317815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 445/881] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:21.323009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3f5a0-f1e8-451c-9557-d5efb1d66fde None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.323300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3f5a0-f1e8-451c-9557-d5efb1d66fde None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.323613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3f5a0-f1e8-451c-9557-d5efb1d66fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.342684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-dff3f5a0-f1e8-451c-9557-d5efb1d66fde None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:21.343895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dff3f5a0-f1e8-451c-9557-d5efb1d66fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:21.393407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:21.419953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 437/882] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:21.562213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82ca76-22a0-4437-b2e1-2d0e69fa72de None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:21.687101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82ca76-22a0-4437-b2e1-2d0e69fa72de None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:21.702063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3f5a0-f1e8-451c-9557-d5efb1d66fde None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:21.728279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad99dfd-49a2-43b5-af28-673b5bb98bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:21.729467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad99dfd-49a2-43b5-af28-673b5bb98bbb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.729754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad99dfd-49a2-43b5-af28-673b5bb98bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.730027 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad99dfd-49a2-43b5-af28-673b5bb98bbb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.730916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad99dfd-49a2-43b5-af28-673b5bb98bbb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:21.732520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad99dfd-49a2-43b5-af28-673b5bb98bbb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-cad99dfd-49a2-43b5-af28-673b5bb98bbb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:21.744398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:21.745460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:21.746357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 446/883] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:31:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:21.752251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad99dfd-49a2-43b5-af28-673b5bb98bbb None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:21.835915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:21.843315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3f5a0-f1e8-451c-9557-d5efb1d66fde None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:21.894685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad99dfd-49a2-43b5-af28-673b5bb98bbb None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:21.898209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2344de9-5274-41ab-98e3-ed25b3ea627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:21.899866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2344de9-5274-41ab-98e3-ed25b3ea627f None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.900706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2344de9-5274-41ab-98e3-ed25b3ea627f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.901512 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2344de9-5274-41ab-98e3-ed25b3ea627f None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.902930 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2344de9-5274-41ab-98e3-ed25b3ea627f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:21.905682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2344de9-5274-41ab-98e3-ed25b3ea627f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:21.935126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c08b5f-5a02-46cc-8d50-e52e6ea646a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:21.935750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c08b5f-5a02-46cc-8d50-e52e6ea646a4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:21.936034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c08b5f-5a02-46cc-8d50-e52e6ea646a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:21.936310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c08b5f-5a02-46cc-8d50-e52e6ea646a4 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:21.937051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c08b5f-5a02-46cc-8d50-e52e6ea646a4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:21.937362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c08b5f-5a02-46cc-8d50-e52e6ea646a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:21.939647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c08b5f-5a02-46cc-8d50-e52e6ea646a4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:21.948323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 447/884] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:31:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:21.954844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c08b5f-5a02-46cc-8d50-e52e6ea646a4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.015436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 438/885] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:31:21 2022] POST /identity/v3/services => generated 269 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:22.037441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a01f1-2b05-4593-bc55-af7a0262e52c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.037740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a01f1-2b05-4593-bc55-af7a0262e52c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.038015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a01f1-2b05-4593-bc55-af7a0262e52c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.043918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 439/886] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:22.045730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cf73c-7b28-4a3b-a6fc-07486278e15c None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.046016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cf73c-7b28-4a3b-a6fc-07486278e15c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.046291 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cf73c-7b28-4a3b-a6fc-07486278e15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.070794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-dd1cf73c-7b28-4a3b-a6fc-07486278e15c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:22.073600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dd1cf73c-7b28-4a3b-a6fc-07486278e15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:22.115696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c08b5f-5a02-46cc-8d50-e52e6ea646a4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.131157 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:22.140504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:22.174918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46137-4793-4cbd-bd62-5a955ac34609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.175591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46137-4793-4cbd-bd62-5a955ac34609 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.175972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46137-4793-4cbd-bd62-5a955ac34609 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.176502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46137-4793-4cbd-bd62-5a955ac34609 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.177235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46137-4793-4cbd-bd62-5a955ac34609 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:22.177843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46137-4793-4cbd-bd62-5a955ac34609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.179659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46137-4793-4cbd-bd62-5a955ac34609 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.194240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 448/888] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:31:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:22.194240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46137-4793-4cbd-bd62-5a955ac34609 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.194786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 440/888] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:22.319691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46137-4793-4cbd-bd62-5a955ac34609 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.357992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ab3e9-a5a8-4688-9331-3b55e60a8331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.358718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ab3e9-a5a8-4688-9331-3b55e60a8331 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.359006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ab3e9-a5a8-4688-9331-3b55e60a8331 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.359283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ab3e9-a5a8-4688-9331-3b55e60a8331 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.360803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ab3e9-a5a8-4688-9331-3b55e60a8331 None admin] RBAC: Adding query filter params (interface=public, service_id=b41218d27dce4d419e905d2b977c14ad, region_id=RegionOne) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:22.361541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ab3e9-a5a8-4688-9331-3b55e60a8331 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.363405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ab3e9-a5a8-4688-9331-3b55e60a8331 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.371926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 449/889] 10.222.0.82 () {56 vars in 1251 bytes} [Wed Aug 24 03:31:22 2022] GET /identity/v3/endpoints?service_id=b41218d27dce4d419e905d2b977c14ad&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:22.377813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cf73c-7b28-4a3b-a6fc-07486278e15c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.489079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ab3e9-a5a8-4688-9331-3b55e60a8331 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.512166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cf73c-7b28-4a3b-a6fc-07486278e15c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.541812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:22.553538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eac7ab-f011-4b91-a3bb-2668ab4481bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.555979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eac7ab-f011-4b91-a3bb-2668ab4481bc None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.555979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eac7ab-f011-4b91-a3bb-2668ab4481bc None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.555979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eac7ab-f011-4b91-a3bb-2668ab4481bc None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.555979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eac7ab-f011-4b91-a3bb-2668ab4481bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.556702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eac7ab-f011-4b91-a3bb-2668ab4481bc None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.573385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 441/890] 10.222.0.82 () {56 vars in 1087 bytes} [Wed Aug 24 03:31:22 2022] GET /identity/v3/services => generated 1268 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:22.574668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff6f57-b51b-44a8-aff9-661f2b53b8a6 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.575173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff6f57-b51b-44a8-aff9-661f2b53b8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.575332 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff6f57-b51b-44a8-aff9-661f2b53b8a6 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.577027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 442/891] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:22.583123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e344e15-69fb-4c99-8886-171d57ae81ab None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.583416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e344e15-69fb-4c99-8886-171d57ae81ab None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.583744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e344e15-69fb-4c99-8886-171d57ae81ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.604724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6e344e15-69fb-4c99-8886-171d57ae81ab None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:22.606027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6e344e15-69fb-4c99-8886-171d57ae81ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:22.632890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ab3e9-a5a8-4688-9331-3b55e60a8331 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.644393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:22.687342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392a11a-94f5-41fc-a1e6-d88a3beecc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.689339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392a11a-94f5-41fc-a1e6-d88a3beecc64 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.690228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392a11a-94f5-41fc-a1e6-d88a3beecc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.690968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392a11a-94f5-41fc-a1e6-d88a3beecc64 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.702959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 443/892] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:22.712201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392a11a-94f5-41fc-a1e6-d88a3beecc64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-a392a11a-94f5-41fc-a1e6-d88a3beecc64 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:22.720577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:22.721816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:22.722708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:22.725000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 450/893] 10.222.0.82 () {56 vars in 1120 bytes} [Wed Aug 24 03:31:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:22.730965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e344e15-69fb-4c99-8886-171d57ae81ab None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.837168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e344e15-69fb-4c99-8886-171d57ae81ab None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:22.874095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d01ab-ddf1-4f80-8041-05aafdb312db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:22.874980 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d01ab-ddf1-4f80-8041-05aafdb312db None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:22.875497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d01ab-ddf1-4f80-8041-05aafdb312db None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:22.875931 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d01ab-ddf1-4f80-8041-05aafdb312db None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:22.876820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d01ab-ddf1-4f80-8041-05aafdb312db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:22.877275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d01ab-ddf1-4f80-8041-05aafdb312db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:22.878820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d01ab-ddf1-4f80-8041-05aafdb312db None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:22.887979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 444/894] 10.222.0.82 () {56 vars in 1115 bytes} [Wed Aug 24 03:31:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:22.898543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392a11a-94f5-41fc-a1e6-d88a3beecc64 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:22.996250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d01ab-ddf1-4f80-8041-05aafdb312db None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:23.008728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392a11a-94f5-41fc-a1e6-d88a3beecc64 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:23.012096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:23.033956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:23.055989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3926c52-1f02-43d7-9056-6c38b4827b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:23.057079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3926c52-1f02-43d7-9056-6c38b4827b9d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.057079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3926c52-1f02-43d7-9056-6c38b4827b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.057233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3926c52-1f02-43d7-9056-6c38b4827b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.070316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3926c52-1f02-43d7-9056-6c38b4827b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:23.072584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3926c52-1f02-43d7-9056-6c38b4827b9d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-a3926c52-1f02-43d7-9056-6c38b4827b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:23.097270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:23.098333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:23.099354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:23.099354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:23.099354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:23.102578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 451/895] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:23.107755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3926c52-1f02-43d7-9056-6c38b4827b9d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:23.153553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d01ab-ddf1-4f80-8041-05aafdb312db None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:23.201149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93207b1c-2ece-48cc-927c-f507f874bd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:23.201813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207b1c-2ece-48cc-927c-f507f874bd6c None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.202089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207b1c-2ece-48cc-927c-f507f874bd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.202318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207b1c-2ece-48cc-927c-f507f874bd6c None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.203019 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93207b1c-2ece-48cc-927c-f507f874bd6c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:23.205060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93207b1c-2ece-48cc-927c-f507f874bd6c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:23.229082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3926c52-1f02-43d7-9056-6c38b4827b9d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:23.264964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767d3b0-63ee-4d69-aae4-c27b297b8596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:23.265601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767d3b0-63ee-4d69-aae4-c27b297b8596 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.265936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767d3b0-63ee-4d69-aae4-c27b297b8596 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.266394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767d3b0-63ee-4d69-aae4-c27b297b8596 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.267084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767d3b0-63ee-4d69-aae4-c27b297b8596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:23.267391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767d3b0-63ee-4d69-aae4-c27b297b8596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:23.268772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767d3b0-63ee-4d69-aae4-c27b297b8596 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:23.283289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 452/896] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:23.289310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767d3b0-63ee-4d69-aae4-c27b297b8596 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:23.309316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:23.320619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 445/897] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:31:22 2022] POST /identity/v3/services => generated 241 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:23.411801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767d3b0-63ee-4d69-aae4-c27b297b8596 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:23.461159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35d84b-67a3-4c51-8ab2-b321edba937e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:23.461868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35d84b-67a3-4c51-8ab2-b321edba937e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.462167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35d84b-67a3-4c51-8ab2-b321edba937e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.462518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35d84b-67a3-4c51-8ab2-b321edba937e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.471071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35d84b-67a3-4c51-8ab2-b321edba937e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:23.472554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35d84b-67a3-4c51-8ab2-b321edba937e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-2a35d84b-67a3-4c51-8ab2-b321edba937e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:23.485981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:23.487038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:23.488564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 453/898] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:31:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:23.494385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93207b1c-2ece-48cc-927c-f507f874bd6c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:23.590861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:23.607305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93207b1c-2ece-48cc-927c-f507f874bd6c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:23.654291 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5794a-5fc6-476e-9d95-616732e6965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:23.654996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5794a-5fc6-476e-9d95-616732e6965e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.655281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5794a-5fc6-476e-9d95-616732e6965e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.655553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5794a-5fc6-476e-9d95-616732e6965e None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.656294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5794a-5fc6-476e-9d95-616732e6965e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:23.657239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5794a-5fc6-476e-9d95-616732e6965e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:23.658905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5794a-5fc6-476e-9d95-616732e6965e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:23.678383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 446/899] 10.222.0.82 () {56 vars in 1149 bytes} [Wed Aug 24 03:31:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:23.683910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35d84b-67a3-4c51-8ab2-b321edba937e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:23.776568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:23.830987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35d84b-67a3-4c51-8ab2-b321edba937e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:23.871136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736433d8-5994-40ca-b5e4-4827a028a6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:23.872120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736433d8-5994-40ca-b5e4-4827a028a6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:23.872641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736433d8-5994-40ca-b5e4-4827a028a6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:23.873120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736433d8-5994-40ca-b5e4-4827a028a6c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:23.953197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736433d8-5994-40ca-b5e4-4827a028a6c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:23.953197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736433d8-5994-40ca-b5e4-4827a028a6c9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:24.038913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:24.059359 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 454/900] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:31:24.065499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5794a-5fc6-476e-9d95-616732e6965e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:24.202282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5794a-5fc6-476e-9d95-616732e6965e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:24.250068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0304-139b-4dc4-af07-3cfc3923ace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:24.251237 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0304-139b-4dc4-af07-3cfc3923ace3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.251466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0304-139b-4dc4-af07-3cfc3923ace3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.251867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0304-139b-4dc4-af07-3cfc3923ace3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.257218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7369ff-e543-4d21-b760-f4e14bd84bcb None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.257697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7369ff-e543-4d21-b760-f4e14bd84bcb None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.257804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7369ff-e543-4d21-b760-f4e14bd84bcb None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.259414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 455/901] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:24.265050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314685-1be5-48a0-bba7-6c2193c6ba8c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.265339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314685-1be5-48a0-bba7-6c2193c6ba8c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.265627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7314685-1be5-48a0-bba7-6c2193c6ba8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.271296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f0304-139b-4dc4-af07-3cfc3923ace3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:24.272990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f0304-139b-4dc4-af07-3cfc3923ace3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:24.286392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-c7314685-1be5-48a0-bba7-6c2193c6ba8c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:24.288404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c7314685-1be5-48a0-bba7-6c2193c6ba8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-d51f0304-139b-4dc4-af07-3cfc3923ace3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:24.291356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:24.292536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:24.293709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:24.293709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:24.293709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:24.293709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 447/902] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:24.294286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:24.306534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0304-139b-4dc4-af07-3cfc3923ace3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:24.394929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 456/903] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:24.437076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0304-139b-4dc4-af07-3cfc3923ace3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:24.474828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702afcdb-1b3a-4ef7-925d-0e9ace84aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:24.475456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702afcdb-1b3a-4ef7-925d-0e9ace84aea3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.475738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702afcdb-1b3a-4ef7-925d-0e9ace84aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.476012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702afcdb-1b3a-4ef7-925d-0e9ace84aea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.476669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702afcdb-1b3a-4ef7-925d-0e9ace84aea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:24.476972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702afcdb-1b3a-4ef7-925d-0e9ace84aea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:24.478453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702afcdb-1b3a-4ef7-925d-0e9ace84aea3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:24.493466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 448/904] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:24.499705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7314685-1be5-48a0-bba7-6c2193c6ba8c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:24.628955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7314685-1be5-48a0-bba7-6c2193c6ba8c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:24.630822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:24.665578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bf490-c26c-4406-8858-b87049266adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:24.666539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf490-c26c-4406-8858-b87049266adb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.667011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf490-c26c-4406-8858-b87049266adb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.667463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf490-c26c-4406-8858-b87049266adb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.674774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bf490-c26c-4406-8858-b87049266adb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:24.676444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bf490-c26c-4406-8858-b87049266adb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-e87bf490-c26c-4406-8858-b87049266adb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:24.688547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:24.689495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:24.690405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:24.690405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:24.690405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:24.690405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:24.690405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:24.690405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:24.690405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:24.690405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:24.692505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 457/905] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:24.698534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702afcdb-1b3a-4ef7-925d-0e9ace84aea3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:24.722486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4cbe5-84a1-4200-9234-ed85babcb5e5 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.722802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4cbe5-84a1-4200-9234-ed85babcb5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.723080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4cbe5-84a1-4200-9234-ed85babcb5e5 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.724414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 458/906] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:24.725111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4cbe5-84a1-4200-9234-ed85babcb5e5 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:24.771008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:24.827771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702afcdb-1b3a-4ef7-925d-0e9ace84aea3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:24.842052 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4cbe5-84a1-4200-9234-ed85babcb5e5 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:24.867568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b70e5d-8767-4de8-9ed1-05ed403e19b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:24.868190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b70e5d-8767-4de8-9ed1-05ed403e19b5 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.868970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b70e5d-8767-4de8-9ed1-05ed403e19b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.869255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b70e5d-8767-4de8-9ed1-05ed403e19b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.869966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b70e5d-8767-4de8-9ed1-05ed403e19b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:24.870469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b70e5d-8767-4de8-9ed1-05ed403e19b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:24.871935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b70e5d-8767-4de8-9ed1-05ed403e19b5 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:24.885843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f355a49-0c5e-4271-9abc-5a90a3dbb9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:24.886177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 449/907] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:24.887125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f355a49-0c5e-4271-9abc-5a90a3dbb9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.887468 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f34fda-3e56-4e79-b7c6-222692534615 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:24.887759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f34fda-3e56-4e79-b7c6-222692534615 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.888039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f34fda-3e56-4e79-b7c6-222692534615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.888276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f355a49-0c5e-4271-9abc-5a90a3dbb9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:24.888778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f355a49-0c5e-4271-9abc-5a90a3dbb9e3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:24.889567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f355a49-0c5e-4271-9abc-5a90a3dbb9e3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:24.891865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f355a49-0c5e-4271-9abc-5a90a3dbb9e3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-6f355a49-0c5e-4271-9abc-5a90a3dbb9e3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:24.902622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:24.903621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:31:24.904560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:24.907474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 459/908] 10.222.0.82 () {56 vars in 1132 bytes} [Wed Aug 24 03:31:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:24.908727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f355a49-0c5e-4271-9abc-5a90a3dbb9e3 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:24.912123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-02f34fda-3e56-4e79-b7c6-222692534615 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:24.913551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-02f34fda-3e56-4e79-b7c6-222692534615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:24.939080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:24.997996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 450/909] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:24.999290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f34fda-3e56-4e79-b7c6-222692534615 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:25.042074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f355a49-0c5e-4271-9abc-5a90a3dbb9e3 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:25.066579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:25.092985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af93a59-b15e-4646-9680-cf11fc2370d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:25.093600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af93a59-b15e-4646-9680-cf11fc2370d0 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.093882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af93a59-b15e-4646-9680-cf11fc2370d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.094595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af93a59-b15e-4646-9680-cf11fc2370d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.095173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af93a59-b15e-4646-9680-cf11fc2370d0 None admin] RBAC: Adding query filter params (role.id=c29a8da689b642ffb03ab69775d7c627, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=b14c2eb2840b46538f169c39f200639f) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:25.095488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af93a59-b15e-4646-9680-cf11fc2370d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:25.096949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af93a59-b15e-4646-9680-cf11fc2370d0 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:25.110995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 460/910] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:24 2022] GET /identity/v3/role_assignments?user.id=b14c2eb2840b46538f169c39f200639f&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=c29a8da689b642ffb03ab69775d7c627 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:25.150393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f34fda-3e56-4e79-b7c6-222692534615 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:25.191977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297428a-deae-4d78-a978-67215d4bc243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:25.193019 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297428a-deae-4d78-a978-67215d4bc243 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.193508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297428a-deae-4d78-a978-67215d4bc243 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.194010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297428a-deae-4d78-a978-67215d4bc243 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.194927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297428a-deae-4d78-a978-67215d4bc243 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:25.195582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297428a-deae-4d78-a978-67215d4bc243 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:25.197218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297428a-deae-4d78-a978-67215d4bc243 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:25.204331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 451/911] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:25.209785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af93a59-b15e-4646-9680-cf11fc2370d0 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:25.290620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297428a-deae-4d78-a978-67215d4bc243 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:25.319904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:25.330746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af93a59-b15e-4646-9680-cf11fc2370d0 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:25.367600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06caecc-d98a-46c5-a7db-a01a6084497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:25.368220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06caecc-d98a-46c5-a7db-a01a6084497f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.368524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06caecc-d98a-46c5-a7db-a01a6084497f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.369187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06caecc-d98a-46c5-a7db-a01a6084497f None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.369418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06caecc-d98a-46c5-a7db-a01a6084497f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:25.369728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06caecc-d98a-46c5-a7db-a01a6084497f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:25.371203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06caecc-d98a-46c5-a7db-a01a6084497f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:25.377707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 461/912] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:25.383239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06caecc-d98a-46c5-a7db-a01a6084497f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:25.407675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297428a-deae-4d78-a978-67215d4bc243 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:25.419669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:25.449688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc64b98-72fb-45f0-b816-72d3494f8c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:25.450844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc64b98-72fb-45f0-b816-72d3494f8c0f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.451355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc64b98-72fb-45f0-b816-72d3494f8c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.451825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc64b98-72fb-45f0-b816-72d3494f8c0f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.452730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc64b98-72fb-45f0-b816-72d3494f8c0f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:25.455027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc64b98-72fb-45f0-b816-72d3494f8c0f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-1fc64b98-72fb-45f0-b816-72d3494f8c0f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:25.465261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:25.466374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:31:25.467263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:25.470261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 452/913] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:31:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:25.476265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc64b98-72fb-45f0-b816-72d3494f8c0f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:25.520007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06caecc-d98a-46c5-a7db-a01a6084497f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:25.561822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2a276-816a-48dd-94c7-86057e0432e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:25.563383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2a276-816a-48dd-94c7-86057e0432e0 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.563883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2a276-816a-48dd-94c7-86057e0432e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.564213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2a276-816a-48dd-94c7-86057e0432e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.565058 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2a276-816a-48dd-94c7-86057e0432e0 None admin] RBAC: Adding query filter params (interface=public, service_id=03887dd4b8fc490bae6da167d57ce3fe, region_id=RegionOne) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:25.565369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2a276-816a-48dd-94c7-86057e0432e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:25.566803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2a276-816a-48dd-94c7-86057e0432e0 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:25.572930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 462/914] 10.222.0.82 () {56 vars in 1251 bytes} [Wed Aug 24 03:31:25 2022] GET /identity/v3/endpoints?service_id=03887dd4b8fc490bae6da167d57ce3fe&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:25.574589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e927e-0b8e-42e0-9b4f-bcf2f08460d4 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.574794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e927e-0b8e-42e0-9b4f-bcf2f08460d4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.575071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e927e-0b8e-42e0-9b4f-bcf2f08460d4 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.576289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 463/915] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:25.581465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e927e-0b8e-42e0-9b4f-bcf2f08460d4 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:25.598596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc64b98-72fb-45f0-b816-72d3494f8c0f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:25.601067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:25.636775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6277fd-d299-49ac-8e0f-e3ad7933de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:25.637399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6277fd-d299-49ac-8e0f-e3ad7933de5e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.637655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6277fd-d299-49ac-8e0f-e3ad7933de5e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.637932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6277fd-d299-49ac-8e0f-e3ad7933de5e None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.638593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6277fd-d299-49ac-8e0f-e3ad7933de5e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:25.638902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6277fd-d299-49ac-8e0f-e3ad7933de5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:25.640272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6277fd-d299-49ac-8e0f-e3ad7933de5e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:25.653045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 453/916] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:31:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:25.653045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0c54c-400d-4c80-b9e7-a699636f4b89 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.653045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0c54c-400d-4c80-b9e7-a699636f4b89 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.653045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0c54c-400d-4c80-b9e7-a699636f4b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.673592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d3d0c54c-400d-4c80-b9e7-a699636f4b89 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:25.674745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d3d0c54c-400d-4c80-b9e7-a699636f4b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:25.706490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:25.713885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e927e-0b8e-42e0-9b4f-bcf2f08460d4 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:25.751742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 454/917] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:25.752914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0c54c-400d-4c80-b9e7-a699636f4b89 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:25.764106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f6f27-5dac-4409-9814-4200bb77e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:25.765374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f6f27-5dac-4409-9814-4200bb77e56e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.765980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f6f27-5dac-4409-9814-4200bb77e56e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.767022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f6f27-5dac-4409-9814-4200bb77e56e None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.767967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f6f27-5dac-4409-9814-4200bb77e56e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:25.769713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f6f27-5dac-4409-9814-4200bb77e56e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:25.784354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 464/918] 10.222.0.82 () {56 vars in 1087 bytes} [Wed Aug 24 03:31:25 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:25.874558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0c54c-400d-4c80-b9e7-a699636f4b89 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:25.912465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28d18a-d1fa-4393-8b0c-d21403e5060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:25.913070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d18a-d1fa-4393-8b0c-d21403e5060b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:25.913348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d18a-d1fa-4393-8b0c-d21403e5060b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:25.913632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d18a-d1fa-4393-8b0c-d21403e5060b None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:25.914681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28d18a-d1fa-4393-8b0c-d21403e5060b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:25.914995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28d18a-d1fa-4393-8b0c-d21403e5060b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:25.916417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28d18a-d1fa-4393-8b0c-d21403e5060b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:25.922906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 455/919] 10.222.0.82 () {56 vars in 1114 bytes} [Wed Aug 24 03:31:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:25.929762 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f6f27-5dac-4409-9814-4200bb77e56e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.033080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:26.036137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28d18a-d1fa-4393-8b0c-d21403e5060b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.053751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f6f27-5dac-4409-9814-4200bb77e56e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.068658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:26.102950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bb3b5-61c1-49a3-8dd3-b1f9c60e07f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.104459 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bb3b5-61c1-49a3-8dd3-b1f9c60e07f6 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.104754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bb3b5-61c1-49a3-8dd3-b1f9c60e07f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.105039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bb3b5-61c1-49a3-8dd3-b1f9c60e07f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.106070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bb3b5-61c1-49a3-8dd3-b1f9c60e07f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.107931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bb3b5-61c1-49a3-8dd3-b1f9c60e07f6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.155057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28d18a-d1fa-4393-8b0c-d21403e5060b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.195831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe28ab-6f36-4ad9-bc21-96a33d0346c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.196693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe28ab-6f36-4ad9-bc21-96a33d0346c2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.197149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe28ab-6f36-4ad9-bc21-96a33d0346c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.197454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe28ab-6f36-4ad9-bc21-96a33d0346c2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.198054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe28ab-6f36-4ad9-bc21-96a33d0346c2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.200335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe28ab-6f36-4ad9-bc21-96a33d0346c2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.207365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 465/920] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-1bfe28ab-6f36-4ad9-bc21-96a33d0346c2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:26.219048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:26.221587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:26.223374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 456/921] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:26.230539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bb3b5-61c1-49a3-8dd3-b1f9c60e07f6 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.377153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bb3b5-61c1-49a3-8dd3-b1f9c60e07f6 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.419181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ed4d7-2d39-470b-ab2c-914311016fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.419812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ed4d7-2d39-470b-ab2c-914311016fcb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.420144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ed4d7-2d39-470b-ab2c-914311016fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.420464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ed4d7-2d39-470b-ab2c-914311016fcb None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.421120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ed4d7-2d39-470b-ab2c-914311016fcb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:26.421432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ed4d7-2d39-470b-ab2c-914311016fcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.423757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ed4d7-2d39-470b-ab2c-914311016fcb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.425278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:26.431033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 466/922] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:26.439485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe28ab-6f36-4ad9-bc21-96a33d0346c2 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.579665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:26.586791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe28ab-6f36-4ad9-bc21-96a33d0346c2 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.625432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfd6d6-895c-4bf4-90ab-949809c28b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.626047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfd6d6-895c-4bf4-90ab-949809c28b1e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.626373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfd6d6-895c-4bf4-90ab-949809c28b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.626720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfd6d6-895c-4bf4-90ab-949809c28b1e None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.627416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfd6d6-895c-4bf4-90ab-949809c28b1e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:26.628115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfd6d6-895c-4bf4-90ab-949809c28b1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.629492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfd6d6-895c-4bf4-90ab-949809c28b1e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.635356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 457/923] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:26.641754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ed4d7-2d39-470b-ab2c-914311016fcb None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.769153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ed4d7-2d39-470b-ab2c-914311016fcb None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.806449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd641512-2647-4221-9a4a-aa8f0210cfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.807130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd641512-2647-4221-9a4a-aa8f0210cfbb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.807412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd641512-2647-4221-9a4a-aa8f0210cfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.807737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd641512-2647-4221-9a4a-aa8f0210cfbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.808450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd641512-2647-4221-9a4a-aa8f0210cfbb None admin] RBAC: Adding query filter params (interface=public, service_id=cffa808d5ccd4f168946e908c74728ae, region_id=RegionOne) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:26.808760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd641512-2647-4221-9a4a-aa8f0210cfbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.810131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd641512-2647-4221-9a4a-aa8f0210cfbb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.816513 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 467/924] 10.222.0.82 () {56 vars in 1251 bytes} [Wed Aug 24 03:31:26 2022] GET /identity/v3/endpoints?service_id=cffa808d5ccd4f168946e908c74728ae&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:26.821996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfd6d6-895c-4bf4-90ab-949809c28b1e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:26.946384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfd6d6-895c-4bf4-90ab-949809c28b1e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:26.985630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ead664-928e-4d4d-8a07-0c25f09fae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:26.986245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ead664-928e-4d4d-8a07-0c25f09fae4e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:26.986540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ead664-928e-4d4d-8a07-0c25f09fae4e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:26.986819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ead664-928e-4d4d-8a07-0c25f09fae4e None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:26.987455 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ead664-928e-4d4d-8a07-0c25f09fae4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:26.989150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ead664-928e-4d4d-8a07-0c25f09fae4e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:26.995862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 458/925] 10.222.0.82 () {56 vars in 1087 bytes} [Wed Aug 24 03:31:26 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:27.191086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4186c4-3dd6-4826-a187-63f6cad6bbd2 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.191450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4186c4-3dd6-4826-a187-63f6cad6bbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.191648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4186c4-3dd6-4826-a187-63f6cad6bbd2 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.193123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 468/926] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:27.198677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499e002-45bb-4d87-b5e6-72cd609f1bd0 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.199199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499e002-45bb-4d87-b5e6-72cd609f1bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.199701 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499e002-45bb-4d87-b5e6-72cd609f1bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.220939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b499e002-45bb-4d87-b5e6-72cd609f1bd0 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:27.222985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b499e002-45bb-4d87-b5e6-72cd609f1bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:27.242767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:27.310003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 459/927] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:27.588571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4186c4-3dd6-4826-a187-63f6cad6bbd2 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:27.662129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:27.715442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4186c4-3dd6-4826-a187-63f6cad6bbd2 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:27.752442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42c7ac-dabf-4a8c-92bb-38b36bc8b304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:27.753325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c7ac-dabf-4a8c-92bb-38b36bc8b304 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.753752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c7ac-dabf-4a8c-92bb-38b36bc8b304 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.754188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c7ac-dabf-4a8c-92bb-38b36bc8b304 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.765029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42c7ac-dabf-4a8c-92bb-38b36bc8b304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-3a42c7ac-dabf-4a8c-92bb-38b36bc8b304 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:27.771603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:27.772785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:27.773874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:27.776501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 469/928] 10.222.0.82 () {56 vars in 1120 bytes} [Wed Aug 24 03:31:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:27.782991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499e002-45bb-4d87-b5e6-72cd609f1bd0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:27.895875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499e002-45bb-4d87-b5e6-72cd609f1bd0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:27.933842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa59a1-bd95-4ac9-813c-b7d2985d50a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:27.934725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa59a1-bd95-4ac9-813c-b7d2985d50a9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.935170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa59a1-bd95-4ac9-813c-b7d2985d50a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.935588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa59a1-bd95-4ac9-813c-b7d2985d50a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.936394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa59a1-bd95-4ac9-813c-b7d2985d50a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:27.936860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa59a1-bd95-4ac9-813c-b7d2985d50a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:27.939412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981715-0ed1-4383-82fa-f65b7cda0bd1 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.939682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa59a1-bd95-4ac9-813c-b7d2985d50a9 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:27.940168 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981715-0ed1-4383-82fa-f65b7cda0bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.940635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981715-0ed1-4383-82fa-f65b7cda0bd1 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.943130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 470/929] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:27.950440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 460/930] 10.222.0.82 () {56 vars in 1115 bytes} [Wed Aug 24 03:31:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:27.951576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaf330-1ba9-49ea-8b6e-06b48004f639 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:27.951864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaf330-1ba9-49ea-8b6e-06b48004f639 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:27.952140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaf330-1ba9-49ea-8b6e-06b48004f639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:27.959511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa59a1-bd95-4ac9-813c-b7d2985d50a9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:27.972583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-56eaf330-1ba9-49ea-8b6e-06b48004f639 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:27.973907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-56eaf330-1ba9-49ea-8b6e-06b48004f639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:27.978072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:28.058926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 471/931] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:28.088531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa59a1-bd95-4ac9-813c-b7d2985d50a9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:28.126561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533e963-1cd2-45d2-b429-ae78d81b214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:28.127234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533e963-1cd2-45d2-b429-ae78d81b214c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:28.127515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533e963-1cd2-45d2-b429-ae78d81b214c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:28.127790 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533e963-1cd2-45d2-b429-ae78d81b214c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:28.139493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533e963-1cd2-45d2-b429-ae78d81b214c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:28.141839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533e963-1cd2-45d2-b429-ae78d81b214c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-a533e963-1cd2-45d2-b429-ae78d81b214c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:28.158390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:28.159630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:28.160832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:28.160832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:28.160832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:28.160832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 461/932] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:28.165518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eaf330-1ba9-49ea-8b6e-06b48004f639 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:28.222228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:28.277051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eaf330-1ba9-49ea-8b6e-06b48004f639 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:28.316160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa66559-a37e-478b-b79d-e670a8e1f901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:28.316847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa66559-a37e-478b-b79d-e670a8e1f901 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:28.317127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa66559-a37e-478b-b79d-e670a8e1f901 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:28.317399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa66559-a37e-478b-b79d-e670a8e1f901 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:28.318051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa66559-a37e-478b-b79d-e670a8e1f901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:28.318357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa66559-a37e-478b-b79d-e670a8e1f901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:28.319804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa66559-a37e-478b-b79d-e670a8e1f901 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:28.333265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 472/933] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:28.338670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533e963-1cd2-45d2-b429-ae78d81b214c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:28.383395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa66559-a37e-478b-b79d-e670a8e1f901 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:28.418153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:28.460851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533e963-1cd2-45d2-b429-ae78d81b214c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:28.499882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e863d7-4899-43e7-82af-6c489538e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:28.501080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e863d7-4899-43e7-82af-6c489538e41a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:28.501368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e863d7-4899-43e7-82af-6c489538e41a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:28.501638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e863d7-4899-43e7-82af-6c489538e41a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:28.510852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e863d7-4899-43e7-82af-6c489538e41a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:28.512331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e863d7-4899-43e7-82af-6c489538e41a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-33e863d7-4899-43e7-82af-6c489538e41a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:28.525038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:28.526063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:28.527086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 462/934] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:31:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:28.527685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b71dcf-3e28-4ff2-a1c0-c2da739f94d0 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:28.527685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b71dcf-3e28-4ff2-a1c0-c2da739f94d0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:28.528175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b71dcf-3e28-4ff2-a1c0-c2da739f94d0 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:28.530552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 463/935] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:28.533001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b71dcf-3e28-4ff2-a1c0-c2da739f94d0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:28.544103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa66559-a37e-478b-b79d-e670a8e1f901 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:28.562821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:28.589612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7d994-d8dd-4b71-a512-8c4d8c971800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:28.590259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7d994-d8dd-4b71-a512-8c4d8c971800 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:28.590538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7d994-d8dd-4b71-a512-8c4d8c971800 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:28.590821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7d994-d8dd-4b71-a512-8c4d8c971800 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:28.591420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7d994-d8dd-4b71-a512-8c4d8c971800 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:28.592921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7d994-d8dd-4b71-a512-8c4d8c971800 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-68b7d994-d8dd-4b71-a512-8c4d8c971800 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:28.603406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:28.604449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:28.605380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 473/936] 10.222.0.82 () {56 vars in 1132 bytes} [Wed Aug 24 03:31:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:28.606775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca19b-8c3a-4d16-abe2-de8a84ee9e63 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:28.606775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca19b-8c3a-4d16-abe2-de8a84ee9e63 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:28.606915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca19b-8c3a-4d16-abe2-de8a84ee9e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:28.626096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9ecca19b-8c3a-4d16-abe2-de8a84ee9e63 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:28.627349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9ecca19b-8c3a-4d16-abe2-de8a84ee9e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:28.667479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b71dcf-3e28-4ff2-a1c0-c2da739f94d0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:28.701862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:28.709772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 474/937] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:28.711035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca19b-8c3a-4d16-abe2-de8a84ee9e63 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:28.721764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ef7c7-02cb-4bb6-b154-caa60ecc959e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:28.722415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ef7c7-02cb-4bb6-b154-caa60ecc959e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:28.722736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ef7c7-02cb-4bb6-b154-caa60ecc959e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:28.723194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ef7c7-02cb-4bb6-b154-caa60ecc959e None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:28.723840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ef7c7-02cb-4bb6-b154-caa60ecc959e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:28.724153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ef7c7-02cb-4bb6-b154-caa60ecc959e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:28.725622 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ef7c7-02cb-4bb6-b154-caa60ecc959e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:28.758685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 464/938] 10.222.0.82 () {56 vars in 1149 bytes} [Wed Aug 24 03:31:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:28.760347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ef7c7-02cb-4bb6-b154-caa60ecc959e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:28.867000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca19b-8c3a-4d16-abe2-de8a84ee9e63 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:28.874220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:28.896117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ef7c7-02cb-4bb6-b154-caa60ecc959e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:28.906184 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183b0fb-e9a0-48f2-b005-d82a7be188ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:28.908322 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183b0fb-e9a0-48f2-b005-d82a7be188ca None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:28.909207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183b0fb-e9a0-48f2-b005-d82a7be188ca None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:28.909668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183b0fb-e9a0-48f2-b005-d82a7be188ca None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:28.910787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183b0fb-e9a0-48f2-b005-d82a7be188ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:28.911280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183b0fb-e9a0-48f2-b005-d82a7be188ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:28.913604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183b0fb-e9a0-48f2-b005-d82a7be188ca None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:28.920423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 475/939] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:28.926754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183b0fb-e9a0-48f2-b005-d82a7be188ca None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:28.941374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc0fa7-fe7d-46c9-ada7-95c94146851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:28.942430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc0fa7-fe7d-46c9-ada7-95c94146851c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:28.942835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc0fa7-fe7d-46c9-ada7-95c94146851c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:28.943181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc0fa7-fe7d-46c9-ada7-95c94146851c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:28.959962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc0fa7-fe7d-46c9-ada7-95c94146851c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:28.962279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc0fa7-fe7d-46c9-ada7-95c94146851c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-35fc0fa7-fe7d-46c9-ada7-95c94146851c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:28.981978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:28.983057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:28.983965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:28.983965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:28.983965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:28.986661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 465/940] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:28.990565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc0fa7-fe7d-46c9-ada7-95c94146851c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:29.027977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:29.049839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183b0fb-e9a0-48f2-b005-d82a7be188ca None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:29.088395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5286-7ff9-4321-b580-9c2dea9b1831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:29.089000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5286-7ff9-4321-b580-9c2dea9b1831 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.089290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5286-7ff9-4321-b580-9c2dea9b1831 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.089561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5286-7ff9-4321-b580-9c2dea9b1831 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.090200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d5286-7ff9-4321-b580-9c2dea9b1831 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:29.090498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d5286-7ff9-4321-b580-9c2dea9b1831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:29.091982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d5286-7ff9-4321-b580-9c2dea9b1831 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:29.097600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 476/941] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:29.098355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5286-7ff9-4321-b580-9c2dea9b1831 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:29.104813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc0fa7-fe7d-46c9-ada7-95c94146851c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:29.133195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:29.144465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe96d8d-dcaa-46a9-bc64-d5d1d42db5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:29.145076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe96d8d-dcaa-46a9-bc64-d5d1d42db5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.145357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe96d8d-dcaa-46a9-bc64-d5d1d42db5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.146499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe96d8d-dcaa-46a9-bc64-d5d1d42db5f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.146499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe96d8d-dcaa-46a9-bc64-d5d1d42db5f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:29.146713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe96d8d-dcaa-46a9-bc64-d5d1d42db5f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:29.148069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe96d8d-dcaa-46a9-bc64-d5d1d42db5f0 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:29.159713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 466/942] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:29.161013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b77f68-ad92-48d8-9298-28fdd1d55986 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.161305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b77f68-ad92-48d8-9298-28fdd1d55986 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.161587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b77f68-ad92-48d8-9298-28fdd1d55986 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.162816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 467/943] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:29.163507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b77f68-ad92-48d8-9298-28fdd1d55986 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:29.208391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5286-7ff9-4321-b580-9c2dea9b1831 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:29.247819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:29.248349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.248635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.248905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.249555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:29.251694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:29.254303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WARNING keystone.server.flask.common [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 03:31:29.260325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:29.267711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b77f68-ad92-48d8-9298-28fdd1d55986 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:29.296841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.sql.core [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 03:31:29.296841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 03:31:29.296841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [parameters: {'id': '9610992b434948178102ea946ebec748', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 03:31:29.296841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81942) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 03:31:29.302690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e85b8-c048-4f86-bc92-37f8b4d287b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:29.303308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e85b8-c048-4f86-bc92-37f8b4d287b9 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.306773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e85b8-c048-4f86-bc92-37f8b4d287b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.306773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e85b8-c048-4f86-bc92-37f8b4d287b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9610992b434948178102ea946ebec748.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9610992b434948178102ea946ebec748. Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:31:29.307172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result.read() Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 03:31:29.308586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application yield resource Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise self.value Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application session.commit() Aug 24 03:31:29.310465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.session.flush() Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application compat.raise_( Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 03:31:29.312249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 03:31:29.314703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result.read() Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:31:29.315716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [parameters: {'id': '9610992b434948178102ea946ebec748', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:29.316844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:29.317874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:29.319081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:29.319081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 03:31:29.319081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 03:31:29.319081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:29.319081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:29.319081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 03:31:29.319081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 03:31:29.319081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9610992b434948178102ea946ebec748. Aug 24 03:31:29.319081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:29.319832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e85b8-c048-4f86-bc92-37f8b4d287b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:29.319933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 477/944] 10.222.0.82 () {60 vars in 1140 bytes} [Wed Aug 24 03:31:29 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:29.320074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e85b8-c048-4f86-bc92-37f8b4d287b9 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:29.320225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:29.401656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 468/945] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:29.403206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd371d-6da8-47df-8c6b-4898fa6a9bb0 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.403606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd371d-6da8-47df-8c6b-4898fa6a9bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.404041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd371d-6da8-47df-8c6b-4898fa6a9bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.411847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:29.423819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83523f-1b07-49a7-8d4e-70c57df9ea04 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:29.444676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-fefd371d-6da8-47df-8c6b-4898fa6a9bb0 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:29.446647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fefd371d-6da8-47df-8c6b-4898fa6a9bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:29.500127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2016d2-cf39-447e-997a-075b07305680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:29.501311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2016d2-cf39-447e-997a-075b07305680 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.501883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2016d2-cf39-447e-997a-075b07305680 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.501883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2016d2-cf39-447e-997a-075b07305680 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.515133 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2016d2-cf39-447e-997a-075b07305680 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:29.518219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2016d2-cf39-447e-997a-075b07305680 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-7c2016d2-cf39-447e-997a-075b07305680 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:29.540109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:29.541325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:29.542878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:29.542878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:29.542878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:29.542878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:29.542878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:29.542878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:29.542878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:29.542878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:29.548895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 478/946] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:29.548895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2016d2-cf39-447e-997a-075b07305680 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:29.578538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 469/947] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:29.578538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd371d-6da8-47df-8c6b-4898fa6a9bb0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:29.685014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2016d2-cf39-447e-997a-075b07305680 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:29.706068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:29.721496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd371d-6da8-47df-8c6b-4898fa6a9bb0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:29.748344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7838e-0c25-4a1e-9709-4b0ebe4dffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:29.749147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7838e-0c25-4a1e-9709-4b0ebe4dffb3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.749526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7838e-0c25-4a1e-9709-4b0ebe4dffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.749925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7838e-0c25-4a1e-9709-4b0ebe4dffb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.764041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7838e-0c25-4a1e-9709-4b0ebe4dffb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:29.765867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7838e-0c25-4a1e-9709-4b0ebe4dffb3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:29.778253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce938f39-f0ef-4b00-8f75-3e0d6936188f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:29.782133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce938f39-f0ef-4b00-8f75-3e0d6936188f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.782133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce938f39-f0ef-4b00-8f75-3e0d6936188f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.782133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce938f39-f0ef-4b00-8f75-3e0d6936188f None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.782133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce938f39-f0ef-4b00-8f75-3e0d6936188f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:29.783028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce938f39-f0ef-4b00-8f75-3e0d6936188f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-e0b7838e-0c25-4a1e-9709-4b0ebe4dffb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:29.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:29.786001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:29.787189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:29.787189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:29.787189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:29.787189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:29.787189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:29.787189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:29.787189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:29.787189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:29.787189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 479/948] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:29.787927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce938f39-f0ef-4b00-8f75-3e0d6936188f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:29.792465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7838e-0c25-4a1e-9709-4b0ebe4dffb3 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:29.802016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 470/949] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:29.807752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:29.808946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce938f39-f0ef-4b00-8f75-3e0d6936188f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:29.910226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7838e-0c25-4a1e-9709-4b0ebe4dffb3 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:29.925909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce938f39-f0ef-4b00-8f75-3e0d6936188f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:29.949641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1f6ae-4730-4b3e-a36d-3bc5add4769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:29.950742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1f6ae-4730-4b3e-a36d-3bc5add4769f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.951299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1f6ae-4730-4b3e-a36d-3bc5add4769f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.951634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1f6ae-4730-4b3e-a36d-3bc5add4769f None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.954622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1f6ae-4730-4b3e-a36d-3bc5add4769f None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:29.954622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1f6ae-4730-4b3e-a36d-3bc5add4769f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:29.954938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1f6ae-4730-4b3e-a36d-3bc5add4769f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:29.968857 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035c90a-fc75-4ed4-a9df-78f5a3e2a01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:29.969634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035c90a-fc75-4ed4-a9df-78f5a3e2a01a None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:29.969634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035c90a-fc75-4ed4-a9df-78f5a3e2a01a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:29.969634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035c90a-fc75-4ed4-a9df-78f5a3e2a01a None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/b14c2eb2840b46538f169c39f200639f/roles/c29a8da689b642ffb03ab69775d7c627` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:29.972373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035c90a-fc75-4ed4-a9df-78f5a3e2a01a None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=b14c2eb2840b46538f169c39f200639f, role_id=c29a8da689b642ffb03ab69775d7c627)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:29.973841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035c90a-fc75-4ed4-a9df-78f5a3e2a01a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:29.986545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 480/950] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:29.986545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1f6ae-4730-4b3e-a36d-3bc5add4769f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:30.077715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:30.080577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 471/951] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:31:29 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/b14c2eb2840b46538f169c39f200639f/roles/c29a8da689b642ffb03ab69775d7c627 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:30.124370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1f6ae-4730-4b3e-a36d-3bc5add4769f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:30.169995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978b91-d3b2-4239-8be3-fa58ef4f0e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:30.171005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978b91-d3b2-4239-8be3-fa58ef4f0e27 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:30.171439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978b91-d3b2-4239-8be3-fa58ef4f0e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:30.171866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978b91-d3b2-4239-8be3-fa58ef4f0e27 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:30.173561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1978b91-d3b2-4239-8be3-fa58ef4f0e27 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:30.175178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1978b91-d3b2-4239-8be3-fa58ef4f0e27 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-d1978b91-d3b2-4239-8be3-fa58ef4f0e27 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:30.185300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:30.186687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:30.187836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:30.190928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 481/952] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:30.197049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035c90a-fc75-4ed4-a9df-78f5a3e2a01a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:30.209885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:30.328540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035c90a-fc75-4ed4-a9df-78f5a3e2a01a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:30.368799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa55fd-38c3-4bcf-b67a-ac5d5e8dd71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:30.369807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa55fd-38c3-4bcf-b67a-ac5d5e8dd71f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:30.370315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa55fd-38c3-4bcf-b67a-ac5d5e8dd71f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:30.370834 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa55fd-38c3-4bcf-b67a-ac5d5e8dd71f None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:30.371682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa55fd-38c3-4bcf-b67a-ac5d5e8dd71f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:30.372197 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa55fd-38c3-4bcf-b67a-ac5d5e8dd71f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:30.373836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa55fd-38c3-4bcf-b67a-ac5d5e8dd71f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:30.380928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 472/953] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:30.387573 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978b91-d3b2-4239-8be3-fa58ef4f0e27 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:30.498524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978b91-d3b2-4239-8be3-fa58ef4f0e27 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:30.534341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:30.538039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53453e4-a38c-4772-8ea3-ce3d09e815eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:30.538687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53453e4-a38c-4772-8ea3-ce3d09e815eb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:30.539025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53453e4-a38c-4772-8ea3-ce3d09e815eb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:30.539266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53453e4-a38c-4772-8ea3-ce3d09e815eb None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:30.540007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53453e4-a38c-4772-8ea3-ce3d09e815eb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:30.540522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53453e4-a38c-4772-8ea3-ce3d09e815eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:30.541777 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53453e4-a38c-4772-8ea3-ce3d09e815eb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:30.547442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 482/954] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:30.555029 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa55fd-38c3-4bcf-b67a-ac5d5e8dd71f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:30.639153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:30.676138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa55fd-38c3-4bcf-b67a-ac5d5e8dd71f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:30.716688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1e1af-8253-4554-bb42-1c8d91905fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:30.717307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1e1af-8253-4554-bb42-1c8d91905fab None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:30.717595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1e1af-8253-4554-bb42-1c8d91905fab None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:30.717873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1e1af-8253-4554-bb42-1c8d91905fab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:30.718576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1e1af-8253-4554-bb42-1c8d91905fab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:30.720506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1e1af-8253-4554-bb42-1c8d91905fab None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:30.826387 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 473/955] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:31.530405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186f76-6592-4e2f-876f-b60b0ca43c8c None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:31.531011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186f76-6592-4e2f-876f-b60b0ca43c8c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:31.531011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186f76-6592-4e2f-876f-b60b0ca43c8c None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:31.533846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 483/956] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:31.539734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c7987-5a9a-48d7-984c-44242941bd43 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:31.540024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c7987-5a9a-48d7-984c-44242941bd43 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:31.540305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c7987-5a9a-48d7-984c-44242941bd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:31.560127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2d1c7987-5a9a-48d7-984c-44242941bd43 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:31.561328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2d1c7987-5a9a-48d7-984c-44242941bd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:31.648163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 474/957] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:31.946673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40186f76-6592-4e2f-876f-b60b0ca43c8c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:32.067173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40186f76-6592-4e2f-876f-b60b0ca43c8c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:32.104534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0875985-8d3d-4edc-8b48-c902c97200c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:32.105181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0875985-8d3d-4edc-8b48-c902c97200c3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.105463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0875985-8d3d-4edc-8b48-c902c97200c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.105740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0875985-8d3d-4edc-8b48-c902c97200c3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.106337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0875985-8d3d-4edc-8b48-c902c97200c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:32.107767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0875985-8d3d-4edc-8b48-c902c97200c3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-a0875985-8d3d-4edc-8b48-c902c97200c3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:32.118085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:32.119340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:31:32.120900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:32.124187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 484/958] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:32.126685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c7987-5a9a-48d7-984c-44242941bd43 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:32.166698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b00aa6-d809-416c-8e51-5568ac25b4bc None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.167026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b00aa6-d809-416c-8e51-5568ac25b4bc None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.167271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b00aa6-d809-416c-8e51-5568ac25b4bc None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.168917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 485/959] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:32.175619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa8986-b1a2-4fd1-ab5f-89265ad38d6c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.176157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa8986-b1a2-4fd1-ab5f-89265ad38d6c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.176637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa8986-b1a2-4fd1-ab5f-89265ad38d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.195843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-fdaa8986-b1a2-4fd1-ab5f-89265ad38d6c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:32.196106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:32.198258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fdaa8986-b1a2-4fd1-ab5f-89265ad38d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:32.254436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c7987-5a9a-48d7-984c-44242941bd43 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:32.259324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:32.275116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 486/960] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:32.276390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18d771-c74b-444a-81e7-dd511998e62f None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.276677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18d771-c74b-444a-81e7-dd511998e62f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.276953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18d771-c74b-444a-81e7-dd511998e62f None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.278549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 487/961] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:32.284121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca92a3-14e5-4ff2-9383-370750c5a465 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.284602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca92a3-14e5-4ff2-9383-370750c5a465 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.285012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca92a3-14e5-4ff2-9383-370750c5a465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.302862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9781-801c-48c0-a300-006a82f5347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:32.303814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9781-801c-48c0-a300-006a82f5347a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.304243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9781-801c-48c0-a300-006a82f5347a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.304658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9781-801c-48c0-a300-006a82f5347a None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.305437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9781-801c-48c0-a300-006a82f5347a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:32.305899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9781-801c-48c0-a300-006a82f5347a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:32.306074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ffca92a3-14e5-4ff2-9383-370750c5a465 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:32.307319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ffca92a3-14e5-4ff2-9383-370750c5a465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:32.307765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9781-801c-48c0-a300-006a82f5347a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:32.315404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 475/962] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:32.322677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9781-801c-48c0-a300-006a82f5347a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:32.408834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 488/963] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:32.457068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9781-801c-48c0-a300-006a82f5347a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:32.473275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:32.495578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb3241-ddde-437b-bd9b-ce5842714489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:32.496427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb3241-ddde-437b-bd9b-ce5842714489 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.496906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb3241-ddde-437b-bd9b-ce5842714489 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.497378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb3241-ddde-437b-bd9b-ce5842714489 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.498151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb3241-ddde-437b-bd9b-ce5842714489 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:32.498630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb3241-ddde-437b-bd9b-ce5842714489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:32.500133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb3241-ddde-437b-bd9b-ce5842714489 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:32.505462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 476/964] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:32.583085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca92a3-14e5-4ff2-9383-370750c5a465 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:32.697837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca92a3-14e5-4ff2-9383-370750c5a465 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:32.715188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb3241-ddde-437b-bd9b-ce5842714489 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:32.720200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:32.745768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:32.746362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.746676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.746959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.747623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:32.749308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:32.814337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.sql.core [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 03:31:32.814337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 03:31:32.814337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [parameters: {'user_id': '5b91046e59c3465b8833ab352bfb11a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 03:31:32.814337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81942) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result.read() Aug 24 03:31:32.823724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:31:32.824728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application yield resource Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise self.value Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application session.commit() Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 03:31:32.825702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.session.flush() Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application compat.raise_( Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 03:31:32.826654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:31:32.827671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application result.read() Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:31:32.829141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [parameters: {'user_id': '5b91046e59c3465b8833ab352bfb11a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:32.830006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 03:31:32.837537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 03:31:32.838585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 03:31:32.838585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 03:31:32.838585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 03:31:32.838585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 03:31:32.838585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 03:31:32.838585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:32.838585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 489/965] 10.222.0.82 () {60 vars in 1131 bytes} [Wed Aug 24 03:31:32 2022] POST /identity/v3/users => generated 160 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:32.838585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:32.842548 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb3241-ddde-437b-bd9b-ce5842714489 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:32.882684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba872c61-454f-4f9f-b999-28c4d4d152e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:32.883634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba872c61-454f-4f9f-b999-28c4d4d152e2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.884095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba872c61-454f-4f9f-b999-28c4d4d152e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.884543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba872c61-454f-4f9f-b999-28c4d4d152e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.898018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba872c61-454f-4f9f-b999-28c4d4d152e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-ba872c61-454f-4f9f-b999-28c4d4d152e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:32.905159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:32.906348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:32.907482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:32.909834 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 477/966] 10.222.0.82 () {56 vars in 1120 bytes} [Wed Aug 24 03:31:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:32.911250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee2b41-55ce-4c07-9f34-7e74a59a4dd8 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.911706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee2b41-55ce-4c07-9f34-7e74a59a4dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:32.912144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee2b41-55ce-4c07-9f34-7e74a59a4dd8 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:32.913618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 478/967] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:32.915827 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee2b41-55ce-4c07-9f34-7e74a59a4dd8 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:32.955842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd518f54-51ac-4e3e-a680-3f990730cb04 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:32.961250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:32.998803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce482e01-2f74-462e-b4fa-84442850052b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:32.999210 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce482e01-2f74-462e-b4fa-84442850052b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:32.999514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce482e01-2f74-462e-b4fa-84442850052b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.003744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce482e01-2f74-462e-b4fa-84442850052b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.013476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce482e01-2f74-462e-b4fa-84442850052b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:33.015393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce482e01-2f74-462e-b4fa-84442850052b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-ce482e01-2f74-462e-b4fa-84442850052b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:33.032972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:33.039845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 490/968] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:31:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cf3e6-2501-4efb-aa6a-bd111d41651f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:33.040816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cf3e6-2501-4efb-aa6a-bd111d41651f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.041581 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cf3e6-2501-4efb-aa6a-bd111d41651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.044089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee2b41-55ce-4c07-9f34-7e74a59a4dd8 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:33.060132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-bd7cf3e6-2501-4efb-aa6a-bd111d41651f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:33.061680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bd7cf3e6-2501-4efb-aa6a-bd111d41651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:33.083657 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b337e38-b561-4753-8867-31b11eceae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:33.084274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b337e38-b561-4753-8867-31b11eceae34 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:33.084558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b337e38-b561-4753-8867-31b11eceae34 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.084845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b337e38-b561-4753-8867-31b11eceae34 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.085991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b337e38-b561-4753-8867-31b11eceae34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:33.086460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b337e38-b561-4753-8867-31b11eceae34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:33.087949 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b337e38-b561-4753-8867-31b11eceae34 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:33.097551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 479/969] 10.222.0.82 () {56 vars in 1115 bytes} [Wed Aug 24 03:31:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:33.098392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b337e38-b561-4753-8867-31b11eceae34 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:33.108649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:33.145024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 491/970] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:33.145807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cf3e6-2501-4efb-aa6a-bd111d41651f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:33.237470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:33.244663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b337e38-b561-4753-8867-31b11eceae34 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:33.273165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cf3e6-2501-4efb-aa6a-bd111d41651f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:33.292822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93199058-05ad-41ba-a8bc-b9160f29be3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:33.293779 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93199058-05ad-41ba-a8bc-b9160f29be3f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:33.294258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93199058-05ad-41ba-a8bc-b9160f29be3f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.294746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93199058-05ad-41ba-a8bc-b9160f29be3f None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.295676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93199058-05ad-41ba-a8bc-b9160f29be3f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:33.296199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93199058-05ad-41ba-a8bc-b9160f29be3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:33.298354 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93199058-05ad-41ba-a8bc-b9160f29be3f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:33.315865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf344d4-21a7-40ef-9ce8-4fbb596f62da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:33.316565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf344d4-21a7-40ef-9ce8-4fbb596f62da None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:33.316847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf344d4-21a7-40ef-9ce8-4fbb596f62da None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.317123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf344d4-21a7-40ef-9ce8-4fbb596f62da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.328460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf344d4-21a7-40ef-9ce8-4fbb596f62da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:33.330055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf344d4-21a7-40ef-9ce8-4fbb596f62da None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:33.346112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 480/971] 10.222.0.82 () {56 vars in 1101 bytes} [Wed Aug 24 03:31:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-fdf344d4-21a7-40ef-9ce8-4fbb596f62da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:33.350779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:33.351788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:33.352742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:33.352742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:33.352742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:33.355110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 492/972] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:33.362350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93199058-05ad-41ba-a8bc-b9160f29be3f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:33.434757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf344d4-21a7-40ef-9ce8-4fbb596f62da None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:33.470079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:33.511229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93199058-05ad-41ba-a8bc-b9160f29be3f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:33.566497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f1829-886d-4c0f-be20-f6ac9f398bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:33.567224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f1829-886d-4c0f-be20-f6ac9f398bec None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:33.567511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f1829-886d-4c0f-be20-f6ac9f398bec None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.567788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f1829-886d-4c0f-be20-f6ac9f398bec None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.568780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f1829-886d-4c0f-be20-f6ac9f398bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:33.569102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f1829-886d-4c0f-be20-f6ac9f398bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:33.570403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f1829-886d-4c0f-be20-f6ac9f398bec None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:33.580402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf344d4-21a7-40ef-9ce8-4fbb596f62da None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:33.587015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 481/973] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:33.593350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f1829-886d-4c0f-be20-f6ac9f398bec None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:33.619323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:33.620681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0f9d8-0b1d-41ff-86a4-966c143ca822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:33.621315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0f9d8-0b1d-41ff-86a4-966c143ca822 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:33.621647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0f9d8-0b1d-41ff-86a4-966c143ca822 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.621928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0f9d8-0b1d-41ff-86a4-966c143ca822 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.622549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0f9d8-0b1d-41ff-86a4-966c143ca822 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:33.624548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0f9d8-0b1d-41ff-86a4-966c143ca822 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-1cc0f9d8-0b1d-41ff-86a4-966c143ca822 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:33.634774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:33.638684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:33.639580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 493/974] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:33.644323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0f9d8-0b1d-41ff-86a4-966c143ca822 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:33.726692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f1829-886d-4c0f-be20-f6ac9f398bec None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:33.765848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:33.787196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378716ac-9f90-4d7b-bdf9-31d152d72dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:33.789759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378716ac-9f90-4d7b-bdf9-31d152d72dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:33.790003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0f9d8-0b1d-41ff-86a4-966c143ca822 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:33.790211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378716ac-9f90-4d7b-bdf9-31d152d72dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.790679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378716ac-9f90-4d7b-bdf9-31d152d72dc7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.805767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378716ac-9f90-4d7b-bdf9-31d152d72dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:33.808889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378716ac-9f90-4d7b-bdf9-31d152d72dc7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-378716ac-9f90-4d7b-bdf9-31d152d72dc7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:33.830194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:33.832031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:31:33.833841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:33.838577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 482/975] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:31:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:33.839709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378716ac-9f90-4d7b-bdf9-31d152d72dc7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:33.843226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64558884-21fb-4225-8977-250f4c7ea592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:33.845311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64558884-21fb-4225-8977-250f4c7ea592 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:33.845602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64558884-21fb-4225-8977-250f4c7ea592 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.845883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64558884-21fb-4225-8977-250f4c7ea592 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.846661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64558884-21fb-4225-8977-250f4c7ea592 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:33.846980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64558884-21fb-4225-8977-250f4c7ea592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:33.848480 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64558884-21fb-4225-8977-250f4c7ea592 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:33.864544 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 494/976] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:33.865819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64558884-21fb-4225-8977-250f4c7ea592 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:33.933314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:33.957846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378716ac-9f90-4d7b-bdf9-31d152d72dc7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:33.995085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6cf3a-004c-40d9-8cc0-4770f87a5ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:33.995691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6cf3a-004c-40d9-8cc0-4770f87a5ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:33.995984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6cf3a-004c-40d9-8cc0-4770f87a5ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:33.996257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6cf3a-004c-40d9-8cc0-4770f87a5ec3 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:33.996997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6cf3a-004c-40d9-8cc0-4770f87a5ec3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:33.997296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6cf3a-004c-40d9-8cc0-4770f87a5ec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:33.998776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6cf3a-004c-40d9-8cc0-4770f87a5ec3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:34.012639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64558884-21fb-4225-8977-250f4c7ea592 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:34.027671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 483/977] 10.222.0.82 () {56 vars in 1149 bytes} [Wed Aug 24 03:31:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:34.034431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6cf3a-004c-40d9-8cc0-4770f87a5ec3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:34.059923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb28d2-0817-486e-86b4-7d501f1c1265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:34.060550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb28d2-0817-486e-86b4-7d501f1c1265 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:34.060842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb28d2-0817-486e-86b4-7d501f1c1265 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:34.061119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb28d2-0817-486e-86b4-7d501f1c1265 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:34.061803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb28d2-0817-486e-86b4-7d501f1c1265 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:34.062114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb28d2-0817-486e-86b4-7d501f1c1265 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:34.064858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb28d2-0817-486e-86b4-7d501f1c1265 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:34.070218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 495/978] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:34.076158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb28d2-0817-486e-86b4-7d501f1c1265 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:34.097348 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:34.161618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6cf3a-004c-40d9-8cc0-4770f87a5ec3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:34.198177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb28d2-0817-486e-86b4-7d501f1c1265 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:34.201244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343f81b-3fa8-4283-9bc7-79af59eee5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:34.201870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f81b-3fa8-4283-9bc7-79af59eee5da None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:34.202149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f81b-3fa8-4283-9bc7-79af59eee5da None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:34.202421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343f81b-3fa8-4283-9bc7-79af59eee5da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:34.206437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:34.238932 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c6e01-a169-443d-aa33-4d38a74cab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:34.239553 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c6e01-a169-443d-aa33-4d38a74cab82 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:34.239831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c6e01-a169-443d-aa33-4d38a74cab82 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:34.240107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c6e01-a169-443d-aa33-4d38a74cab82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:34.240818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c6e01-a169-443d-aa33-4d38a74cab82 None admin] RBAC: Adding query filter params (interface=admin, service_id=cffa808d5ccd4f168946e908c74728ae, region_id=RegionOne) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:34.241139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c6e01-a169-443d-aa33-4d38a74cab82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:34.242632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c6e01-a169-443d-aa33-4d38a74cab82 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:34.253540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 496/979] 10.222.0.82 () {56 vars in 1249 bytes} [Wed Aug 24 03:31:34 2022] GET /identity/v3/endpoints?service_id=cffa808d5ccd4f168946e908c74728ae&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:34.255082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c6e01-a169-443d-aa33-4d38a74cab82 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:34.284179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343f81b-3fa8-4283-9bc7-79af59eee5da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:34.285672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343f81b-3fa8-4283-9bc7-79af59eee5da None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:34.349560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:34.378769 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c6e01-a169-443d-aa33-4d38a74cab82 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:34.415128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 484/980] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:34 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:31:34.420245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3af3a-b706-4f78-aa17-e6884ee118df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:34.420727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3af3a-b706-4f78-aa17-e6884ee118df None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:34.421010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3af3a-b706-4f78-aa17-e6884ee118df None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:34.421189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343f81b-3fa8-4283-9bc7-79af59eee5da None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:34.421289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3af3a-b706-4f78-aa17-e6884ee118df None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:34.421889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3af3a-b706-4f78-aa17-e6884ee118df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:34.423709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3af3a-b706-4f78-aa17-e6884ee118df None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:34.442559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 497/981] 10.222.0.82 () {56 vars in 1087 bytes} [Wed Aug 24 03:31:34 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:34.473025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:34.526474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d9c2a-886b-4a15-938b-cefbfd244d8f None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:34.526801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d9c2a-886b-4a15-938b-cefbfd244d8f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:34.527082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d9c2a-886b-4a15-938b-cefbfd244d8f None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:34.528614 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 498/982] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:34.535115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69566c4-4a12-440c-bd02-954823822af2 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:34.535465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69566c4-4a12-440c-bd02-954823822af2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:34.535681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69566c4-4a12-440c-bd02-954823822af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:34.553421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343f81b-3fa8-4283-9bc7-79af59eee5da None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:34.556170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-f69566c4-4a12-440c-bd02-954823822af2 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:34.557524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f69566c4-4a12-440c-bd02-954823822af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:34.616539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285a8b3-b570-484d-9595-e851cacdf48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:34.617106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285a8b3-b570-484d-9595-e851cacdf48d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:34.617522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285a8b3-b570-484d-9595-e851cacdf48d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:34.617794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285a8b3-b570-484d-9595-e851cacdf48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:34.630403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285a8b3-b570-484d-9595-e851cacdf48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:34.631920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285a8b3-b570-484d-9595-e851cacdf48d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:34.641830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 499/983] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-e285a8b3-b570-484d-9595-e851cacdf48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:34.652395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:34.653424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:34.654495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:34.654495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:34.654495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:34.654495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 485/984] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:34.660360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69566c4-4a12-440c-bd02-954823822af2 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:34.740149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:34.772708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69566c4-4a12-440c-bd02-954823822af2 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:34.809483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55685bb7-f13b-4e32-b459-4d65ff03f1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:34.810101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55685bb7-f13b-4e32-b459-4d65ff03f1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:34.810387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55685bb7-f13b-4e32-b459-4d65ff03f1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:34.810708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55685bb7-f13b-4e32-b459-4d65ff03f1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:34.811369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55685bb7-f13b-4e32-b459-4d65ff03f1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:34.811675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55685bb7-f13b-4e32-b459-4d65ff03f1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:34.813646 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55685bb7-f13b-4e32-b459-4d65ff03f1c3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:34.830458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 500/985] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:34.831746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285a8b3-b570-484d-9595-e851cacdf48d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:34.913917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55685bb7-f13b-4e32-b459-4d65ff03f1c3 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:34.928317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:34.945519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285a8b3-b570-484d-9595-e851cacdf48d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:34.985863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269ec7c-6d0b-4f63-8d9d-015e100d9b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:34.986528 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269ec7c-6d0b-4f63-8d9d-015e100d9b08 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:34.986814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269ec7c-6d0b-4f63-8d9d-015e100d9b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:34.987090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269ec7c-6d0b-4f63-8d9d-015e100d9b08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:34.994808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269ec7c-6d0b-4f63-8d9d-015e100d9b08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:34.996266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a269ec7c-6d0b-4f63-8d9d-015e100d9b08 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-a269ec7c-6d0b-4f63-8d9d-015e100d9b08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:35.012101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:35.013117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:35.014453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:35.014453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:35.014453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:35.014453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:35.014453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:35.014453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:35.014453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:35.014453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:35.014453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 486/986] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:35.022416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269ec7c-6d0b-4f63-8d9d-015e100d9b08 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:35.046164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55685bb7-f13b-4e32-b459-4d65ff03f1c3 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:35.078066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:35.087313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d90fbf-0332-42f4-b511-6511b521cfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:35.088198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d90fbf-0332-42f4-b511-6511b521cfb1 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:35.088679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d90fbf-0332-42f4-b511-6511b521cfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:35.089130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d90fbf-0332-42f4-b511-6511b521cfb1 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:35.089990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d90fbf-0332-42f4-b511-6511b521cfb1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:35.091472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d90fbf-0332-42f4-b511-6511b521cfb1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:35.147999 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269ec7c-6d0b-4f63-8d9d-015e100d9b08 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:35.194057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f47385-141a-4f6a-b121-8deb81340e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:35.194788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f47385-141a-4f6a-b121-8deb81340e8b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:35.195161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f47385-141a-4f6a-b121-8deb81340e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:35.195527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f47385-141a-4f6a-b121-8deb81340e8b None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:35.196380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f47385-141a-4f6a-b121-8deb81340e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:35.196779 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f47385-141a-4f6a-b121-8deb81340e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:35.198357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f47385-141a-4f6a-b121-8deb81340e8b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:35.211927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 501/987] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:31:34 2022] POST /identity/v3/services => generated 242 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:35.222149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 487/988] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:35.230833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d90fbf-0332-42f4-b511-6511b521cfb1 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:35.386011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d90fbf-0332-42f4-b511-6511b521cfb1 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:35.427349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a760d0-af80-421f-9543-e8ff2a78da83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:35.427966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a760d0-af80-421f-9543-e8ff2a78da83 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:35.428242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a760d0-af80-421f-9543-e8ff2a78da83 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:35.428515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a760d0-af80-421f-9543-e8ff2a78da83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:35.429286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a760d0-af80-421f-9543-e8ff2a78da83 None admin] RBAC: Adding query filter params (role.id=c29a8da689b642ffb03ab69775d7c627, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=b14c2eb2840b46538f169c39f200639f) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:35.429599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a760d0-af80-421f-9543-e8ff2a78da83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:35.431088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a760d0-af80-421f-9543-e8ff2a78da83 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:35.442820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 502/989] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:35 2022] GET /identity/v3/role_assignments?user.id=b14c2eb2840b46538f169c39f200639f&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=c29a8da689b642ffb03ab69775d7c627 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:36.434360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447daba7-76fe-4f6d-97ca-4afb741aff18 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:36.434945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447daba7-76fe-4f6d-97ca-4afb741aff18 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:36.435038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447daba7-76fe-4f6d-97ca-4afb741aff18 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:36.436479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 488/990] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:36.441627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2a815-3380-46d3-a419-e3635b29d24d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:36.442044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2a815-3380-46d3-a419-e3635b29d24d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:36.442264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2a815-3380-46d3-a419-e3635b29d24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:36.461585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-7fe2a815-3380-46d3-a419-e3635b29d24d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:36.462813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7fe2a815-3380-46d3-a419-e3635b29d24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:36.518175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:36.540340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 503/991] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:36.807504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447daba7-76fe-4f6d-97ca-4afb741aff18 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:36.817565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:36.912823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447daba7-76fe-4f6d-97ca-4afb741aff18 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:36.949093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697e243-4d0b-42fa-bd48-5642935e3062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:36.949729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697e243-4d0b-42fa-bd48-5642935e3062 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:36.950009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697e243-4d0b-42fa-bd48-5642935e3062 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:36.950278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697e243-4d0b-42fa-bd48-5642935e3062 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:36.950888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697e243-4d0b-42fa-bd48-5642935e3062 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:36.952257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697e243-4d0b-42fa-bd48-5642935e3062 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-f697e243-4d0b-42fa-bd48-5642935e3062 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:36.961193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:36.962253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:31:36.963174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:36.966834 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 489/992] 10.222.0.82 () {56 vars in 1126 bytes} [Wed Aug 24 03:31:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:36.968310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2a815-3380-46d3-a419-e3635b29d24d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:37.094895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2a815-3380-46d3-a419-e3635b29d24d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:37.131091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba993d-02fa-4a5c-b998-e4445abbd8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:37.131714 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba993d-02fa-4a5c-b998-e4445abbd8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:37.131998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba993d-02fa-4a5c-b998-e4445abbd8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:37.132274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba993d-02fa-4a5c-b998-e4445abbd8e9 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:37.132922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba993d-02fa-4a5c-b998-e4445abbd8e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:37.133231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba993d-02fa-4a5c-b998-e4445abbd8e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:37.134645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba993d-02fa-4a5c-b998-e4445abbd8e9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:37.145185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 504/993] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:37.146286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697e243-4d0b-42fa-bd48-5642935e3062 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:37.238239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:37.279193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605cd4a-c2c9-42d0-9582-4c8bdbd2344f None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:37.279816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605cd4a-c2c9-42d0-9582-4c8bdbd2344f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:37.280089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605cd4a-c2c9-42d0-9582-4c8bdbd2344f None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:37.281782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 505/994] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:37.287189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697e243-4d0b-42fa-bd48-5642935e3062 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:37.287481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcc989-e55f-4f16-88bd-70d4bb4c8c3b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:37.287767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcc989-e55f-4f16-88bd-70d4bb4c8c3b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:37.288041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcc989-e55f-4f16-88bd-70d4bb4c8c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:37.306920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-18bcc989-e55f-4f16-88bd-70d4bb4c8c3b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:37.308123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-18bcc989-e55f-4f16-88bd-70d4bb4c8c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:37.326166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248b4f8-2f6e-4420-8c5f-ba98ad8ec402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:37.326815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248b4f8-2f6e-4420-8c5f-ba98ad8ec402 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:37.327098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248b4f8-2f6e-4420-8c5f-ba98ad8ec402 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:37.327376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248b4f8-2f6e-4420-8c5f-ba98ad8ec402 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:37.328032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248b4f8-2f6e-4420-8c5f-ba98ad8ec402 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:37.328335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248b4f8-2f6e-4420-8c5f-ba98ad8ec402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:37.330320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248b4f8-2f6e-4420-8c5f-ba98ad8ec402 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:37.335328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 490/995] 10.222.0.82 () {56 vars in 1122 bytes} [Wed Aug 24 03:31:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:37.340810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248b4f8-2f6e-4420-8c5f-ba98ad8ec402 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:37.360165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:37.391599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 506/996] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:37.447322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248b4f8-2f6e-4420-8c5f-ba98ad8ec402 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:37.462017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c0ef7-a535-46ed-be4b-dceea9e9ea22 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:37.462618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c0ef7-a535-46ed-be4b-dceea9e9ea22 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:37.462618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c0ef7-a535-46ed-be4b-dceea9e9ea22 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:37.464106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 507/997] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:37.469701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d0e61-6906-4832-8343-0f8e9b4d557d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:37.470181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d0e61-6906-4832-8343-0f8e9b4d557d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:37.470626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d0e61-6906-4832-8343-0f8e9b4d557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:37.484269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7b764-a9a6-4a8a-8fbc-2bae0561caf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:37.485637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7b764-a9a6-4a8a-8fbc-2bae0561caf5 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:37.486073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7b764-a9a6-4a8a-8fbc-2bae0561caf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:37.486262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7b764-a9a6-4a8a-8fbc-2bae0561caf5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:37.486967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7b764-a9a6-4a8a-8fbc-2bae0561caf5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:37.488349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7b764-a9a6-4a8a-8fbc-2bae0561caf5 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:37.489731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-283d0e61-6906-4832-8343-0f8e9b4d557d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:37.490974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-283d0e61-6906-4832-8343-0f8e9b4d557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:37.493766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:37.563623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 508/998] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:37.587659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 491/999] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:37.666033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d0e61-6906-4832-8343-0f8e9b4d557d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:37.788727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d0e61-6906-4832-8343-0f8e9b4d557d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:37.798664 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:37.844826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de88af-c410-4cec-bd29-d5f2336e67bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:37.845948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de88af-c410-4cec-bd29-d5f2336e67bb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:37.846464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de88af-c410-4cec-bd29-d5f2336e67bb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:37.847034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de88af-c410-4cec-bd29-d5f2336e67bb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:37.847652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de88af-c410-4cec-bd29-d5f2336e67bb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:37.849095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de88af-c410-4cec-bd29-d5f2336e67bb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:37.851163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7b764-a9a6-4a8a-8fbc-2bae0561caf5 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-20de88af-c410-4cec-bd29-d5f2336e67bb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:37.862498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:37.863723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:37.870357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 509/1000] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:37.871468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de88af-c410-4cec-bd29-d5f2336e67bb None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:37.965484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7b764-a9a6-4a8a-8fbc-2bae0561caf5 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:37.993998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de88af-c410-4cec-bd29-d5f2336e67bb None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:38.003689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9713514-cdc1-4a37-9fd1-c7d18dd1238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:38.004384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9713514-cdc1-4a37-9fd1-c7d18dd1238e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:38.004665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9713514-cdc1-4a37-9fd1-c7d18dd1238e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:38.004942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9713514-cdc1-4a37-9fd1-c7d18dd1238e None admin] PATH_INFO: `/v3/services/image` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:38.005539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9713514-cdc1-4a37-9fd1-c7d18dd1238e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:38.006960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9713514-cdc1-4a37-9fd1-c7d18dd1238e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-a9713514-cdc1-4a37-9fd1-c7d18dd1238e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:38.016646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:38.017704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:38.019613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 492/1001] 10.222.0.82 () {56 vars in 1105 bytes} [Wed Aug 24 03:31:37 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:38.023654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9713514-cdc1-4a37-9fd1-c7d18dd1238e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:38.057998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ddae5-e915-4f26-823c-d1885c417a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:38.058645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ddae5-e915-4f26-823c-d1885c417a69 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:38.058938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ddae5-e915-4f26-823c-d1885c417a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:38.059215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ddae5-e915-4f26-823c-d1885c417a69 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:38.059858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ddae5-e915-4f26-823c-d1885c417a69 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:38.060166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ddae5-e915-4f26-823c-d1885c417a69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:38.061552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ddae5-e915-4f26-823c-d1885c417a69 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:38.070930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 510/1002] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:38.073314 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ddae5-e915-4f26-823c-d1885c417a69 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:38.089674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:38.111261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:38.137070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9713514-cdc1-4a37-9fd1-c7d18dd1238e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:38.174467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e610c-0aeb-4523-89e9-06da5878cbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:38.180323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e610c-0aeb-4523-89e9-06da5878cbba None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:38.180323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e610c-0aeb-4523-89e9-06da5878cbba None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:38.180323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e610c-0aeb-4523-89e9-06da5878cbba None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:38.180323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e610c-0aeb-4523-89e9-06da5878cbba None admin] RBAC: Adding query filter params (name=image) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:38.180323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e610c-0aeb-4523-89e9-06da5878cbba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:38.180323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e610c-0aeb-4523-89e9-06da5878cbba None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:38.188335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 493/1003] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:38.189984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e610c-0aeb-4523-89e9-06da5878cbba None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:38.198105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ddae5-e915-4f26-823c-d1885c417a69 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:38.252276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3f710-2350-4e7c-8820-0447e6a97119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:38.252841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3f710-2350-4e7c-8820-0447e6a97119 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:38.254225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3f710-2350-4e7c-8820-0447e6a97119 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:38.254225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3f710-2350-4e7c-8820-0447e6a97119 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:38.255140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3f710-2350-4e7c-8820-0447e6a97119 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:38.255140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3f710-2350-4e7c-8820-0447e6a97119 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:38.256337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3f710-2350-4e7c-8820-0447e6a97119 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:38.262379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 511/1004] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:38.271875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3f710-2350-4e7c-8820-0447e6a97119 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:38.330862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e610c-0aeb-4523-89e9-06da5878cbba None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:38.359312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:38.380787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b4ef5-350d-4235-85a6-80ca0450cc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:38.381512 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b4ef5-350d-4235-85a6-80ca0450cc44 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:38.381792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b4ef5-350d-4235-85a6-80ca0450cc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:38.382062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b4ef5-350d-4235-85a6-80ca0450cc44 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:38.382745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b4ef5-350d-4235-85a6-80ca0450cc44 None admin] RBAC: Adding query filter params (type=image) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:38.383113 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b4ef5-350d-4235-85a6-80ca0450cc44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:38.385055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b4ef5-350d-4235-85a6-80ca0450cc44 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:38.387856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:38.394191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 494/1005] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:38.397238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3f710-2350-4e7c-8820-0447e6a97119 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:38.439597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb5c76-e782-465d-be6b-25c1af25c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:38.440685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb5c76-e782-465d-be6b-25c1af25c863 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:38.441190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb5c76-e782-465d-be6b-25c1af25c863 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:38.441684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb5c76-e782-465d-be6b-25c1af25c863 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:38.442740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb5c76-e782-465d-be6b-25c1af25c863 None admin] RBAC: Adding query filter params (interface=public, service_id=4723ee89add1412aa58b197bbf9599bb, region_id=RegionOne) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:38.443310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb5c76-e782-465d-be6b-25c1af25c863 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:38.444985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb5c76-e782-465d-be6b-25c1af25c863 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:38.462064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 512/1006] 10.222.0.82 () {56 vars in 1251 bytes} [Wed Aug 24 03:31:38 2022] GET /identity/v3/endpoints?service_id=4723ee89add1412aa58b197bbf9599bb&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:38.463350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b4ef5-350d-4235-85a6-80ca0450cc44 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:38.589256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b4ef5-350d-4235-85a6-80ca0450cc44 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:38.632706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ced2f0-6f6b-406d-88e6-51b9e4e6ec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:38.633324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ced2f0-6f6b-406d-88e6-51b9e4e6ec60 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:38.633606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ced2f0-6f6b-406d-88e6-51b9e4e6ec60 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:38.633881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ced2f0-6f6b-406d-88e6-51b9e4e6ec60 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:38.634489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ced2f0-6f6b-406d-88e6-51b9e4e6ec60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:38.635885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ced2f0-6f6b-406d-88e6-51b9e4e6ec60 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:38.643038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 495/1007] 10.222.0.82 () {56 vars in 1087 bytes} [Wed Aug 24 03:31:38 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:39.628233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d3a46-44f8-4ad2-80d4-77a8121ac6bd None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:39.628764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d3a46-44f8-4ad2-80d4-77a8121ac6bd None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:39.628764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d3a46-44f8-4ad2-80d4-77a8121ac6bd None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:39.630275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 513/1008] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:39.635633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a214d0-8819-44a5-a6b1-eafdecbb02ee None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:39.636084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a214d0-8819-44a5-a6b1-eafdecbb02ee None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:39.636185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a214d0-8819-44a5-a6b1-eafdecbb02ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:39.654948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b8a214d0-8819-44a5-a6b1-eafdecbb02ee None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:39.656196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b8a214d0-8819-44a5-a6b1-eafdecbb02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:39.706265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:39.738677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 496/1009] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:40.027407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d3a46-44f8-4ad2-80d4-77a8121ac6bd None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:40.167962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d3a46-44f8-4ad2-80d4-77a8121ac6bd None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:40.204931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a5f23-6846-44e5-9352-4ea17563240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:40.205577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a5f23-6846-44e5-9352-4ea17563240e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:40.205849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a5f23-6846-44e5-9352-4ea17563240e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:40.206123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a5f23-6846-44e5-9352-4ea17563240e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:40.207320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a5f23-6846-44e5-9352-4ea17563240e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:40.208870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a5f23-6846-44e5-9352-4ea17563240e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:40.210825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 514/1010] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:40.216929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a214d0-8819-44a5-a6b1-eafdecbb02ee None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:40.346501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a214d0-8819-44a5-a6b1-eafdecbb02ee None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:40.351824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918344a8-dd7f-47b1-92d3-f8efd2793bb3 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:40.352108 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918344a8-dd7f-47b1-92d3-f8efd2793bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:40.352385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918344a8-dd7f-47b1-92d3-f8efd2793bb3 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:40.353931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 515/1011] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:40.359227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35613ca-5c25-4e4a-b9c5-c37b97aa0b67 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:40.359525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35613ca-5c25-4e4a-b9c5-c37b97aa0b67 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:40.359877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35613ca-5c25-4e4a-b9c5-c37b97aa0b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:40.392064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a35613ca-5c25-4e4a-b9c5-c37b97aa0b67 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:40.393379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a35613ca-5c25-4e4a-b9c5-c37b97aa0b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:40.394394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2ce21-04a4-4f7f-ba02-d622e40f3d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:40.395034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2ce21-04a4-4f7f-ba02-d622e40f3d68 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:40.395355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2ce21-04a4-4f7f-ba02-d622e40f3d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:40.395636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2ce21-04a4-4f7f-ba02-d622e40f3d68 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:40.396283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2ce21-04a4-4f7f-ba02-d622e40f3d68 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:40.397719 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2ce21-04a4-4f7f-ba02-d622e40f3d68 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:40.421304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:40.467968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 516/1012] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:40.502562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 497/1013] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:31:40 2022] POST /identity/v3/projects => generated 311 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:40.815797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b77ad-6dd8-48b6-92c0-734f97de458d None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:40.816383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b77ad-6dd8-48b6-92c0-734f97de458d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:40.816723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b77ad-6dd8-48b6-92c0-734f97de458d None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:40.818966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 517/1014] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:40.825255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09901d35-40d5-42ea-9faf-55228e21a587 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:40.825864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09901d35-40d5-42ea-9faf-55228e21a587 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:40.825864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09901d35-40d5-42ea-9faf-55228e21a587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:40.847313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-09901d35-40d5-42ea-9faf-55228e21a587 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:40.850997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-09901d35-40d5-42ea-9faf-55228e21a587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:40.855331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b77ad-6dd8-48b6-92c0-734f97de458d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:40.892989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:40.946115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 498/1015] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:41.015175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b77ad-6dd8-48b6-92c0-734f97de458d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:41.088384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e634d9-a2ab-4415-b1c6-f343802baf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:41.088896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e634d9-a2ab-4415-b1c6-f343802baf66 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:41.089242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e634d9-a2ab-4415-b1c6-f343802baf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:41.089517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e634d9-a2ab-4415-b1c6-f343802baf66 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:41.090358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e634d9-a2ab-4415-b1c6-f343802baf66 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:41.092087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e634d9-a2ab-4415-b1c6-f343802baf66 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:41.193403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 518/1016] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:31:40 2022] POST /identity/v3/services => generated 242 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:41.293957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09901d35-40d5-42ea-9faf-55228e21a587 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:41.460107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09901d35-40d5-42ea-9faf-55228e21a587 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:41.521320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a2661-cff7-4526-948a-49e8df4f2835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:41.521978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a2661-cff7-4526-948a-49e8df4f2835 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:41.522265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a2661-cff7-4526-948a-49e8df4f2835 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:41.522570 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a2661-cff7-4526-948a-49e8df4f2835 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:41.523160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a2661-cff7-4526-948a-49e8df4f2835 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:41.524551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a2661-cff7-4526-948a-49e8df4f2835 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-4d0a2661-cff7-4526-948a-49e8df4f2835 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:41.542956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:41.544393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:41.545262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 499/1017] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:41.552308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e634d9-a2ab-4415-b1c6-f343802baf66 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:41.578764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:41.692621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e634d9-a2ab-4415-b1c6-f343802baf66 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:41.733920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e023b-ea01-4b4a-bebc-49117a6fa4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:41.734457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e023b-ea01-4b4a-bebc-49117a6fa4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:41.734835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e023b-ea01-4b4a-bebc-49117a6fa4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:41.735758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e023b-ea01-4b4a-bebc-49117a6fa4c6 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:41.736832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e023b-ea01-4b4a-bebc-49117a6fa4c6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:41.738199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e023b-ea01-4b4a-bebc-49117a6fa4c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:41.739363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e023b-ea01-4b4a-bebc-49117a6fa4c6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:41.746787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 519/1018] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:41.753123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a2661-cff7-4526-948a-49e8df4f2835 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:41.783208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:41.876066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a2661-cff7-4526-948a-49e8df4f2835 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:41.913185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5e0df-fd25-481a-ac2a-a20a4224425a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:41.913817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5e0df-fd25-481a-ac2a-a20a4224425a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:41.914101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5e0df-fd25-481a-ac2a-a20a4224425a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:41.914378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5e0df-fd25-481a-ac2a-a20a4224425a None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:41.915100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5e0df-fd25-481a-ac2a-a20a4224425a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:41.915404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5e0df-fd25-481a-ac2a-a20a4224425a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:41.916830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5e0df-fd25-481a-ac2a-a20a4224425a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:41.923523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 500/1019] 10.222.0.82 () {56 vars in 1112 bytes} [Wed Aug 24 03:31:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:41.929217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e023b-ea01-4b4a-bebc-49117a6fa4c6 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:42.029307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:42.068422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e023b-ea01-4b4a-bebc-49117a6fa4c6 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:42.118001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744ba2e-43cc-4f61-87c9-aa0d77b68fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:42.119109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ba2e-43cc-4f61-87c9-aa0d77b68fdf None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:42.119631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ba2e-43cc-4f61-87c9-aa0d77b68fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:42.120140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ba2e-43cc-4f61-87c9-aa0d77b68fdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:42.121387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ba2e-43cc-4f61-87c9-aa0d77b68fdf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:42.124044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ba2e-43cc-4f61-87c9-aa0d77b68fdf None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:42.226575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 520/1020] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:42.770160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be9ad1-7dfa-4968-870f-aae7ee7c5da8 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:42.770435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be9ad1-7dfa-4968-870f-aae7ee7c5da8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:42.770890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be9ad1-7dfa-4968-870f-aae7ee7c5da8 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:42.772353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 501/1021] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:42.777859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe92218-1946-475c-8af5-a2e4a8d0d798 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:42.778142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe92218-1946-475c-8af5-a2e4a8d0d798 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:42.778415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe92218-1946-475c-8af5-a2e4a8d0d798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:42.797946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-8fe92218-1946-475c-8af5-a2e4a8d0d798 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:42.799259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8fe92218-1946-475c-8af5-a2e4a8d0d798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:42.878541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 521/1022] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:43.163926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be9ad1-7dfa-4968-870f-aae7ee7c5da8 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:43.226314 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647ffe8-fc40-49d0-a483-012e88f4ac5e None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:43.226886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647ffe8-fc40-49d0-a483-012e88f4ac5e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:43.226886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647ffe8-fc40-49d0-a483-012e88f4ac5e None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:43.228250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 522/1023] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:43.233543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed231800-21fa-4bb6-9a45-2433976abc5c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:43.233827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed231800-21fa-4bb6-9a45-2433976abc5c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:43.234441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed231800-21fa-4bb6-9a45-2433976abc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:43.253162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ed231800-21fa-4bb6-9a45-2433976abc5c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:43.254361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ed231800-21fa-4bb6-9a45-2433976abc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:43.263038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:43.280276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be9ad1-7dfa-4968-870f-aae7ee7c5da8 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:43.314895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:43.321739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8253d1-a4a0-41ec-b688-146fff63f6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:43.322380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8253d1-a4a0-41ec-b688-146fff63f6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:43.322696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8253d1-a4a0-41ec-b688-146fff63f6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:43.322973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8253d1-a4a0-41ec-b688-146fff63f6b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:43.324106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8253d1-a4a0-41ec-b688-146fff63f6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:43.325647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8253d1-a4a0-41ec-b688-146fff63f6b8 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:43.330813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 523/1025] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:43.331137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 502/1025] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:43.333890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed231800-21fa-4bb6-9a45-2433976abc5c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:43.440692 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed231800-21fa-4bb6-9a45-2433976abc5c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:43.476789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87476af8-2ea0-4639-9315-1d9e432c8736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:43.477405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87476af8-2ea0-4639-9315-1d9e432c8736 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:43.477689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87476af8-2ea0-4639-9315-1d9e432c8736 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:43.477971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87476af8-2ea0-4639-9315-1d9e432c8736 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:43.478662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87476af8-2ea0-4639-9315-1d9e432c8736 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:43.480100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87476af8-2ea0-4639-9315-1d9e432c8736 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:43.592492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8253d1-a4a0-41ec-b688-146fff63f6b8 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:43.602630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 524/1026] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:31:43 2022] POST /identity/v3/users => generated 285 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:31:43.708448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8253d1-a4a0-41ec-b688-146fff63f6b8 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:43.747805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecfb5b-bacf-45c2-b48d-bb54d0a1c869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:43.748692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecfb5b-bacf-45c2-b48d-bb54d0a1c869 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:43.748692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecfb5b-bacf-45c2-b48d-bb54d0a1c869 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:43.748971 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecfb5b-bacf-45c2-b48d-bb54d0a1c869 None admin] PATH_INFO: `/v3/services/image` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:43.749552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecfb5b-bacf-45c2-b48d-bb54d0a1c869 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:43.751635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecfb5b-bacf-45c2-b48d-bb54d0a1c869 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-11ecfb5b-bacf-45c2-b48d-bb54d0a1c869 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:43.761649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:43.762940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:43.763989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 503/1027] 10.222.0.82 () {56 vars in 1105 bytes} [Wed Aug 24 03:31:43 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:43.769597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87476af8-2ea0-4639-9315-1d9e432c8736 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:43.827596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:43.887634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87476af8-2ea0-4639-9315-1d9e432c8736 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:43.924768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea96a52-5177-44dd-8b97-0d07d459e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:43.925246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea96a52-5177-44dd-8b97-0d07d459e15b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:43.925531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea96a52-5177-44dd-8b97-0d07d459e15b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:43.925809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea96a52-5177-44dd-8b97-0d07d459e15b None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:43.926448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea96a52-5177-44dd-8b97-0d07d459e15b None admin] RBAC: Adding query filter params (name=image) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:43.926789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea96a52-5177-44dd-8b97-0d07d459e15b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:43.928173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea96a52-5177-44dd-8b97-0d07d459e15b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:43.933649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 525/1028] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:43.939624 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecfb5b-bacf-45c2-b48d-bb54d0a1c869 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:43.960723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:44.061561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecfb5b-bacf-45c2-b48d-bb54d0a1c869 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:44.097449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0d36-840c-4977-ae93-d01bc961e6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:44.098062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0d36-840c-4977-ae93-d01bc961e6ac None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.098350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0d36-840c-4977-ae93-d01bc961e6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.098646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0d36-840c-4977-ae93-d01bc961e6ac None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.099283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0d36-840c-4977-ae93-d01bc961e6ac None admin] RBAC: Adding query filter params (type=image) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:44.099590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0d36-840c-4977-ae93-d01bc961e6ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:44.100963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0d36-840c-4977-ae93-d01bc961e6ac None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:44.112075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 504/1029] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:44.113262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea96a52-5177-44dd-8b97-0d07d459e15b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:44.219832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea96a52-5177-44dd-8b97-0d07d459e15b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:44.233213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:44.255323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4ea91-928e-45cf-972b-76b1c7b37feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:44.255939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4ea91-928e-45cf-972b-76b1c7b37feb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.256218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4ea91-928e-45cf-972b-76b1c7b37feb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.256486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4ea91-928e-45cf-972b-76b1c7b37feb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.257193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4ea91-928e-45cf-972b-76b1c7b37feb None admin] RBAC: Adding query filter params (interface=public, service_id=42bbc823a7ed409bb0b935d68d98fe0f, region_id=RegionOne) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:44.257500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4ea91-928e-45cf-972b-76b1c7b37feb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:44.258893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4ea91-928e-45cf-972b-76b1c7b37feb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:44.264269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 526/1030] 10.222.0.82 () {56 vars in 1251 bytes} [Wed Aug 24 03:31:44 2022] GET /identity/v3/endpoints?service_id=42bbc823a7ed409bb0b935d68d98fe0f&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:44.270937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0d36-840c-4977-ae93-d01bc961e6ac None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:44.281366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc05944-c2ec-4af7-8140-2e40ef82f65f None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.281746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc05944-c2ec-4af7-8140-2e40ef82f65f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.281964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc05944-c2ec-4af7-8140-2e40ef82f65f None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.283453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 527/1031] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:44.288861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e27d55-8543-40f9-9049-88719eaff93e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.289483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e27d55-8543-40f9-9049-88719eaff93e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.289650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e27d55-8543-40f9-9049-88719eaff93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.308223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-30e27d55-8543-40f9-9049-88719eaff93e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:44.309428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-30e27d55-8543-40f9-9049-88719eaff93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:44.366030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:44.384420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 528/1032] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:44.385285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0d36-840c-4977-ae93-d01bc961e6ac None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:44.431930 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cde573-8e7c-468a-89b1-1c6fdb8de91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:44.432566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cde573-8e7c-468a-89b1-1c6fdb8de91b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.432882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cde573-8e7c-468a-89b1-1c6fdb8de91b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.433169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cde573-8e7c-468a-89b1-1c6fdb8de91b None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.433819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cde573-8e7c-468a-89b1-1c6fdb8de91b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:44.435243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cde573-8e7c-468a-89b1-1c6fdb8de91b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:44.448333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 505/1033] 10.222.0.82 () {56 vars in 1087 bytes} [Wed Aug 24 03:31:44 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:31:44.660119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e27d55-8543-40f9-9049-88719eaff93e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:44.769209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e27d55-8543-40f9-9049-88719eaff93e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:44.778031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:44.810292 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107b714-a6c3-4bf9-b416-fb9fef3cf526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:44.810987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b714-a6c3-4bf9-b416-fb9fef3cf526 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.811276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b714-a6c3-4bf9-b416-fb9fef3cf526 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.811549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b714-a6c3-4bf9-b416-fb9fef3cf526 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.823923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107b714-a6c3-4bf9-b416-fb9fef3cf526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-7107b714-a6c3-4bf9-b416-fb9fef3cf526 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:44.829557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:44.830856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:44.832141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:44.834587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 529/1034] 10.222.0.82 () {56 vars in 1120 bytes} [Wed Aug 24 03:31:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:44.836692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cde573-8e7c-468a-89b1-1c6fdb8de91b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:44.945213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cde573-8e7c-468a-89b1-1c6fdb8de91b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:44.982856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0c73a-45db-4315-94e1-20401f66cb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:44.983467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0c73a-45db-4315-94e1-20401f66cb36 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:44.983755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0c73a-45db-4315-94e1-20401f66cb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:44.984032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0c73a-45db-4315-94e1-20401f66cb36 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:44.984832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0c73a-45db-4315-94e1-20401f66cb36 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:44.985737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0c73a-45db-4315-94e1-20401f66cb36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:44.987145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0c73a-45db-4315-94e1-20401f66cb36 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:44.996939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 506/1035] 10.222.0.82 () {56 vars in 1115 bytes} [Wed Aug 24 03:31:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:45.003610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107b714-a6c3-4bf9-b416-fb9fef3cf526 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.121150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107b714-a6c3-4bf9-b416-fb9fef3cf526 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.158776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d811d-3442-4926-9dc5-981c51784c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.159424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d811d-3442-4926-9dc5-981c51784c08 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.159707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d811d-3442-4926-9dc5-981c51784c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.159985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d811d-3442-4926-9dc5-981c51784c08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.173613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d811d-3442-4926-9dc5-981c51784c08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.175125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d811d-3442-4926-9dc5-981c51784c08 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-732d811d-3442-4926-9dc5-981c51784c08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:45.190547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:45.191567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:45.192505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:45.192505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:45.192505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:45.194870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 530/1036] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:45.197568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0c73a-45db-4315-94e1-20401f66cb36 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.210037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:45.305225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0c73a-45db-4315-94e1-20401f66cb36 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.345502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ceee1-5f42-4c4a-b943-9390a740600f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.346119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ceee1-5f42-4c4a-b943-9390a740600f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.346404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ceee1-5f42-4c4a-b943-9390a740600f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.346704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ceee1-5f42-4c4a-b943-9390a740600f None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.347401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ceee1-5f42-4c4a-b943-9390a740600f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:45.347707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ceee1-5f42-4c4a-b943-9390a740600f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.349098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ceee1-5f42-4c4a-b943-9390a740600f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.366830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 507/1037] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:45.368460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d811d-3442-4926-9dc5-981c51784c08 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.388979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:45.475104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d811d-3442-4926-9dc5-981c51784c08 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.520919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716df0e7-e925-48dc-96ca-cc86ca6af57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.521569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716df0e7-e925-48dc-96ca-cc86ca6af57e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.521855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716df0e7-e925-48dc-96ca-cc86ca6af57e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.522133 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716df0e7-e925-48dc-96ca-cc86ca6af57e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.529684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716df0e7-e925-48dc-96ca-cc86ca6af57e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.531184 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716df0e7-e925-48dc-96ca-cc86ca6af57e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-716df0e7-e925-48dc-96ca-cc86ca6af57e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:45.543344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:45.544384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:45.545404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 531/1038] 10.222.0.82 () {56 vars in 1093 bytes} [Wed Aug 24 03:31:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:45.552359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ceee1-5f42-4c4a-b943-9390a740600f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.627798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:45.666861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ceee1-5f42-4c4a-b943-9390a740600f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.667819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db975097-a858-4b4e-8c7c-219997db08ef None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.668322 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db975097-a858-4b4e-8c7c-219997db08ef None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.668811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db975097-a858-4b4e-8c7c-219997db08ef None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.673019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 532/1039] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:45.679280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dd25f-8da4-47fc-980b-29dbf46af1b0 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.679679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dd25f-8da4-47fc-980b-29dbf46af1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.680031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dd25f-8da4-47fc-980b-29dbf46af1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.701597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6d7dd25f-8da4-47fc-980b-29dbf46af1b0 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:45.704628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6d7dd25f-8da4-47fc-980b-29dbf46af1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:45.708154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4f7ea-1fdc-404e-8744-78bbf0f3e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.708769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4f7ea-1fdc-404e-8744-78bbf0f3e375 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.709060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4f7ea-1fdc-404e-8744-78bbf0f3e375 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.709335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4f7ea-1fdc-404e-8744-78bbf0f3e375 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.710076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4f7ea-1fdc-404e-8744-78bbf0f3e375 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:45.710386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4f7ea-1fdc-404e-8744-78bbf0f3e375 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.711873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4f7ea-1fdc-404e-8744-78bbf0f3e375 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:45.730244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 508/1040] 10.222.0.82 () {56 vars in 1133 bytes} [Wed Aug 24 03:31:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:45.736465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4f7ea-1fdc-404e-8744-78bbf0f3e375 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:45.781383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 533/1041] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:45.851166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4f7ea-1fdc-404e-8744-78bbf0f3e375 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:45.882079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:45.902095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a0ca6-6b19-4cc8-9c9a-d43b26c550fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:45.902812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a0ca6-6b19-4cc8-9c9a-d43b26c550fe None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:45.903146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a0ca6-6b19-4cc8-9c9a-d43b26c550fe None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:45.903424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a0ca6-6b19-4cc8-9c9a-d43b26c550fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:45.983217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a0ca6-6b19-4cc8-9c9a-d43b26c550fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:45.985200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a0ca6-6b19-4cc8-9c9a-d43b26c550fe None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.057644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7dd25f-8da4-47fc-980b-29dbf46af1b0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.077203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:46.131700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 509/1042] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:31:46.137466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a0ca6-6b19-4cc8-9c9a-d43b26c550fe None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.165496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:46.192663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7dd25f-8da4-47fc-980b-29dbf46af1b0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.236772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36825ee4-843a-4728-a2dd-88ea2d07be0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.237970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36825ee4-843a-4728-a2dd-88ea2d07be0b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.237970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36825ee4-843a-4728-a2dd-88ea2d07be0b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.237970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36825ee4-843a-4728-a2dd-88ea2d07be0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.252968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36825ee4-843a-4728-a2dd-88ea2d07be0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-36825ee4-843a-4728-a2dd-88ea2d07be0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:46.257843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:46.259306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:46.261496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 534/1043] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:46.266343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36825ee4-843a-4728-a2dd-88ea2d07be0b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.289162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a0ca6-6b19-4cc8-9c9a-d43b26c550fe None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.345188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85f3af-5ded-42b4-a98a-35ddf38acb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.346462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85f3af-5ded-42b4-a98a-35ddf38acb5c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.346462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85f3af-5ded-42b4-a98a-35ddf38acb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.346462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85f3af-5ded-42b4-a98a-35ddf38acb5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.353542 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:46.358348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85f3af-5ded-42b4-a98a-35ddf38acb5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:46.361183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85f3af-5ded-42b4-a98a-35ddf38acb5c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-ee85f3af-5ded-42b4-a98a-35ddf38acb5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:46.386261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:46.387498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:46.388762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:46.388762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:46.388762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:46.388762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 510/1044] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:46.396115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85f3af-5ded-42b4-a98a-35ddf38acb5c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.420818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36825ee4-843a-4728-a2dd-88ea2d07be0b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.476875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82de90-bcfc-40ea-aa53-1b6c5a67fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.477614 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82de90-bcfc-40ea-aa53-1b6c5a67fe3e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.477903 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82de90-bcfc-40ea-aa53-1b6c5a67fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.478181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82de90-bcfc-40ea-aa53-1b6c5a67fe3e None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.478889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82de90-bcfc-40ea-aa53-1b6c5a67fe3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:46.479196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82de90-bcfc-40ea-aa53-1b6c5a67fe3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:46.480590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82de90-bcfc-40ea-aa53-1b6c5a67fe3e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.490278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 535/1045] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:46.491503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9bb59-89ae-48da-84ba-8a1aaf319963 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.491792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9bb59-89ae-48da-84ba-8a1aaf319963 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.492065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9bb59-89ae-48da-84ba-8a1aaf319963 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.493422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 536/1046] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:46.498727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f2723-2792-4ee2-abcd-2eabbdba2c70 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.499019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f2723-2792-4ee2-abcd-2eabbdba2c70 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.499298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f2723-2792-4ee2-abcd-2eabbdba2c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.504384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:46.517948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-fb5f2723-2792-4ee2-abcd-2eabbdba2c70 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:46.519185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fb5f2723-2792-4ee2-abcd-2eabbdba2c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:46.533516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85f3af-5ded-42b4-a98a-35ddf38acb5c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.572378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe2b8a-8961-4cd9-aea5-029b8327ca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.572985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe2b8a-8961-4cd9-aea5-029b8327ca49 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.573265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe2b8a-8961-4cd9-aea5-029b8327ca49 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.573538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe2b8a-8961-4cd9-aea5-029b8327ca49 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.574188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe2b8a-8961-4cd9-aea5-029b8327ca49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:46.574491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe2b8a-8961-4cd9-aea5-029b8327ca49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:46.575893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe2b8a-8961-4cd9-aea5-029b8327ca49 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.588883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 511/1047] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:46.590080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe2b8a-8961-4cd9-aea5-029b8327ca49 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.612505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 537/1048] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:46.614006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f2723-2792-4ee2-abcd-2eabbdba2c70 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.698919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:46.721980 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe2b8a-8961-4cd9-aea5-029b8327ca49 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.764714 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f2723-2792-4ee2-abcd-2eabbdba2c70 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.771934 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd8cf2-5d96-4769-ba51-600786f7dba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.771934 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8cf2-5d96-4769-ba51-600786f7dba7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.771934 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8cf2-5d96-4769-ba51-600786f7dba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.772561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8cf2-5d96-4769-ba51-600786f7dba7 None admin] PATH_INFO: `/v3/users/79ed6a0bf1364c269e43ae7e83dfec89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.783442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:46.796182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd8cf2-5d96-4769-ba51-600786f7dba7 None admin] RBAC: Authorizing `identity:get_user(user_id=79ed6a0bf1364c269e43ae7e83dfec89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:46.798722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd8cf2-5d96-4769-ba51-600786f7dba7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.800400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 512/1049] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/users/79ed6a0bf1364c269e43ae7e83dfec89 => generated 285 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:46.806982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd8cf2-5d96-4769-ba51-600786f7dba7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.833617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0961f73-7bdb-41fa-a4d7-8b04ddace930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.834265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0961f73-7bdb-41fa-a4d7-8b04ddace930 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.834571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0961f73-7bdb-41fa-a4d7-8b04ddace930 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.834869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0961f73-7bdb-41fa-a4d7-8b04ddace930 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:46.842720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0961f73-7bdb-41fa-a4d7-8b04ddace930 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:46.844869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0961f73-7bdb-41fa-a4d7-8b04ddace930 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-f0961f73-7bdb-41fa-a4d7-8b04ddace930 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:46.859326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:46.860918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:46.863206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:46.863206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:46.863206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:46.863206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:46.863206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:46.863206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:46.863206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:46.863206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:46.863206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 538/1050] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:46.867486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0961f73-7bdb-41fa-a4d7-8b04ddace930 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:46.951110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd8cf2-5d96-4769-ba51-600786f7dba7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:46.996988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee92d3c-3cee-4db3-aef3-3398035a2928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:46.997843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee92d3c-3cee-4db3-aef3-3398035a2928 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:46.998250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee92d3c-3cee-4db3-aef3-3398035a2928 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:46.998747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee92d3c-3cee-4db3-aef3-3398035a2928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:47.002406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0961f73-7bdb-41fa-a4d7-8b04ddace930 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:47.031781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:47.048773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfccc21-c59a-4321-a61b-c76ae4ab75ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:47.051489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfccc21-c59a-4321-a61b-c76ae4ab75ec None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:47.051489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfccc21-c59a-4321-a61b-c76ae4ab75ec None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:47.051489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfccc21-c59a-4321-a61b-c76ae4ab75ec None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:47.051489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfccc21-c59a-4321-a61b-c76ae4ab75ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:47.051489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfccc21-c59a-4321-a61b-c76ae4ab75ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:47.052403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfccc21-c59a-4321-a61b-c76ae4ab75ec None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:47.071640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 539/1051] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:47.072934 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfccc21-c59a-4321-a61b-c76ae4ab75ec None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:47.080974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:47.089661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee92d3c-3cee-4db3-aef3-3398035a2928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:47.091816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee92d3c-3cee-4db3-aef3-3398035a2928 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:47.189992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfccc21-c59a-4321-a61b-c76ae4ab75ec None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:47.217191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 513/1052] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:31:47.217191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee92d3c-3cee-4db3-aef3-3398035a2928 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:47.230359 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d34429-0384-41c3-a1a6-7dec0d2f7b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:47.231052 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d34429-0384-41c3-a1a6-7dec0d2f7b30 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:47.231332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d34429-0384-41c3-a1a6-7dec0d2f7b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:47.231607 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d34429-0384-41c3-a1a6-7dec0d2f7b30 None admin] PATH_INFO: `/v3/services/image` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:47.232203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d34429-0384-41c3-a1a6-7dec0d2f7b30 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:47.233935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d34429-0384-41c3-a1a6-7dec0d2f7b30 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-e7d34429-0384-41c3-a1a6-7dec0d2f7b30 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:47.245763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:47.247055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:47.247948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 540/1053] 10.222.0.82 () {56 vars in 1105 bytes} [Wed Aug 24 03:31:47 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:47.249278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d34429-0384-41c3-a1a6-7dec0d2f7b30 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:47.301156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:47.331966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:47.338385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee92d3c-3cee-4db3-aef3-3398035a2928 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:47.363699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d34429-0384-41c3-a1a6-7dec0d2f7b30 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:47.383333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce1786-38d4-46c9-af32-c8de5b5b9504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:47.383938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce1786-38d4-46c9-af32-c8de5b5b9504 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:47.384242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce1786-38d4-46c9-af32-c8de5b5b9504 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:47.384485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce1786-38d4-46c9-af32-c8de5b5b9504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:47.385228 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce1786-38d4-46c9-af32-c8de5b5b9504 None admin] RBAC: Adding query filter params (role.id=c29a8da689b642ffb03ab69775d7c627, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=0f34fe3fe912498b90dbf950e7fc999d) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:47.385540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce1786-38d4-46c9-af32-c8de5b5b9504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:47.387056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce1786-38d4-46c9-af32-c8de5b5b9504 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:47.394216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 514/1054] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:47 2022] GET /identity/v3/role_assignments?user.id=0f34fe3fe912498b90dbf950e7fc999d&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=c29a8da689b642ffb03ab69775d7c627 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:47.394924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce1786-38d4-46c9-af32-c8de5b5b9504 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:47.406221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb45a8c-6f69-41c4-8ff7-813771885f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:47.407028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb45a8c-6f69-41c4-8ff7-813771885f2f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:47.407365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb45a8c-6f69-41c4-8ff7-813771885f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:47.407627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb45a8c-6f69-41c4-8ff7-813771885f2f None admin] PATH_INFO: `/v3/projects/a405a5ff63284c4ebc649d18a620ff38` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:47.408873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb45a8c-6f69-41c4-8ff7-813771885f2f None admin] RBAC: Authorizing `identity:get_project(project_id=a405a5ff63284c4ebc649d18a620ff38)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:47.410412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb45a8c-6f69-41c4-8ff7-813771885f2f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:47.412347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 541/1055] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:31:47 2022] GET /identity/v3/projects/a405a5ff63284c4ebc649d18a620ff38 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:47.418852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb45a8c-6f69-41c4-8ff7-813771885f2f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:47.538331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce1786-38d4-46c9-af32-c8de5b5b9504 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:47.560054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb45a8c-6f69-41c4-8ff7-813771885f2f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:47.579211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:47.582734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cf9eb-78bd-4478-9393-0856238d9873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:47.583360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cf9eb-78bd-4478-9393-0856238d9873 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:47.583963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cf9eb-78bd-4478-9393-0856238d9873 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:47.583963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cf9eb-78bd-4478-9393-0856238d9873 None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:47.584537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cf9eb-78bd-4478-9393-0856238d9873 None admin] RBAC: Adding query filter params (name=image) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:47.584839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cf9eb-78bd-4478-9393-0856238d9873 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:47.586236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cf9eb-78bd-4478-9393-0856238d9873 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:47.593187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 515/1056] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:47.599267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cf9eb-78bd-4478-9393-0856238d9873 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:47.610787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8a996-27e1-44a1-9ef5-2e7327219e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:47.610787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8a996-27e1-44a1-9ef5-2e7327219e5b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:47.610787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8a996-27e1-44a1-9ef5-2e7327219e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:47.610787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8a996-27e1-44a1-9ef5-2e7327219e5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:47.611246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8a996-27e1-44a1-9ef5-2e7327219e5b None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=a405a5ff63284c4ebc649d18a620ff38, user.id=79ed6a0bf1364c269e43ae7e83dfec89) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:47.611525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8a996-27e1-44a1-9ef5-2e7327219e5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:47.613467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8a996-27e1-44a1-9ef5-2e7327219e5b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:47.630541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 542/1057] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:47 2022] GET /identity/v3/role_assignments?user.id=79ed6a0bf1364c269e43ae7e83dfec89&scope.project.id=a405a5ff63284c4ebc649d18a620ff38&role.id=9e904216086e4a06abf9216b0c863f89 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:47.643150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:47.752336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cf9eb-78bd-4478-9393-0856238d9873 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:47.801334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c095f-6826-4bc2-b7d9-a7b117c3554c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:47.801811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c095f-6826-4bc2-b7d9-a7b117c3554c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:47.802099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c095f-6826-4bc2-b7d9-a7b117c3554c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:47.802356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c095f-6826-4bc2-b7d9-a7b117c3554c None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:47.803041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c095f-6826-4bc2-b7d9-a7b117c3554c None admin] RBAC: Adding query filter params (type=image) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:47.803341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c095f-6826-4bc2-b7d9-a7b117c3554c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:47.804724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c095f-6826-4bc2-b7d9-a7b117c3554c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:47.810869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 516/1058] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:47.818291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8a996-27e1-44a1-9ef5-2e7327219e5b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:47.955041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8a996-27e1-44a1-9ef5-2e7327219e5b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:47.997872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08984fe-a96f-426a-9163-10e5a7aae4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:47.998524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08984fe-a96f-426a-9163-10e5a7aae4a4 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:47.998813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08984fe-a96f-426a-9163-10e5a7aae4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:47.999092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08984fe-a96f-426a-9163-10e5a7aae4a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:48.003422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08984fe-a96f-426a-9163-10e5a7aae4a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:48.003422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08984fe-a96f-426a-9163-10e5a7aae4a4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:48.099155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 543/1059] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:31:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:49.410530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43f31b-1d01-40cf-affc-3d34650f0d6c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.411054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43f31b-1d01-40cf-affc-3d34650f0d6c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.411325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43f31b-1d01-40cf-affc-3d34650f0d6c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.412561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 517/1060] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:49.417855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd2a9b-381b-40fd-91ae-04da4544ffc3 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.418260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd2a9b-381b-40fd-91ae-04da4544ffc3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.418493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd2a9b-381b-40fd-91ae-04da4544ffc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.443326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-cacd2a9b-381b-40fd-91ae-04da4544ffc3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:49.444641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-cacd2a9b-381b-40fd-91ae-04da4544ffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:49.480630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:49.528713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 544/1061] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:49.588216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa9be3-7a17-4c29-86c6-98122aff2aba None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.588525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa9be3-7a17-4c29-86c6-98122aff2aba None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.588776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa9be3-7a17-4c29-86c6-98122aff2aba None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.590256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 518/1062] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:49.595993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672c723-3d09-4b13-81d4-2b4ba46ca080 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.596283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672c723-3d09-4b13-81d4-2b4ba46ca080 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.596558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672c723-3d09-4b13-81d4-2b4ba46ca080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.618144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-7672c723-3d09-4b13-81d4-2b4ba46ca080 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:49.619367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7672c723-3d09-4b13-81d4-2b4ba46ca080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:49.693011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 545/1063] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:49.796031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa9be3-7a17-4c29-86c6-98122aff2aba None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:49.896940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:49.912378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa9be3-7a17-4c29-86c6-98122aff2aba None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:49.969133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad594eb8-67e1-46ea-aa57-46d8051b7249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:49.970679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad594eb8-67e1-46ea-aa57-46d8051b7249 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:49.970679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad594eb8-67e1-46ea-aa57-46d8051b7249 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:49.970890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad594eb8-67e1-46ea-aa57-46d8051b7249 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:49.974490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672c723-3d09-4b13-81d4-2b4ba46ca080 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:49.990790 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad594eb8-67e1-46ea-aa57-46d8051b7249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-ad594eb8-67e1-46ea-aa57-46d8051b7249 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:49.996366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:49.997510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:49.998402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 519/1064] 10.222.0.82 () {56 vars in 1120 bytes} [Wed Aug 24 03:31:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:50.004613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad594eb8-67e1-46ea-aa57-46d8051b7249 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.119381 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.128202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad594eb8-67e1-46ea-aa57-46d8051b7249 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.137474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672c723-3d09-4b13-81d4-2b4ba46ca080 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.167681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b72baa-d356-463e-8fec-a0f4bfef11be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.168295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b72baa-d356-463e-8fec-a0f4bfef11be None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.168582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b72baa-d356-463e-8fec-a0f4bfef11be None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.168858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b72baa-d356-463e-8fec-a0f4bfef11be None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.169523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b72baa-d356-463e-8fec-a0f4bfef11be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:50.169838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b72baa-d356-463e-8fec-a0f4bfef11be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.171263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b72baa-d356-463e-8fec-a0f4bfef11be None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.173803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb9248-000a-4480-aec2-77397f55e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.174436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb9248-000a-4480-aec2-77397f55e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.174746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb9248-000a-4480-aec2-77397f55e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.175023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb9248-000a-4480-aec2-77397f55e7f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.180988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 520/1065] 10.222.0.82 () {56 vars in 1115 bytes} [Wed Aug 24 03:31:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:50.182266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace9f98-9f7c-4c28-b94d-a21ed000eb74 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.182589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace9f98-9f7c-4c28-b94d-a21ed000eb74 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.182875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace9f98-9f7c-4c28-b94d-a21ed000eb74 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.184251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 521/1066] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:50.185295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb9248-000a-4480-aec2-77397f55e7f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:50.187349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace9f98-9f7c-4c28-b94d-a21ed000eb74 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-fbcb9248-000a-4480-aec2-77397f55e7f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:50.190048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:50.191046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:50.191938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 546/1067] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:50.192690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ceb48-0b24-4498-884f-66f2420edc4a None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.192690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ceb48-0b24-4498-884f-66f2420edc4a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.192820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ceb48-0b24-4498-884f-66f2420edc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.204260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.210256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ec7ceb48-0b24-4498-884f-66f2420edc4a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:50.211485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ec7ceb48-0b24-4498-884f-66f2420edc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:50.278414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 547/1068] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:50.279204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ceb48-0b24-4498-884f-66f2420edc4a None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.297916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace9f98-9f7c-4c28-b94d-a21ed000eb74 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.337823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe616f9-fbdb-4a70-9b5d-5d827d41da23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.338458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe616f9-fbdb-4a70-9b5d-5d827d41da23 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.338833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe616f9-fbdb-4a70-9b5d-5d827d41da23 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.339230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe616f9-fbdb-4a70-9b5d-5d827d41da23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.351210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe616f9-fbdb-4a70-9b5d-5d827d41da23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.352674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe616f9-fbdb-4a70-9b5d-5d827d41da23 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-2fe616f9-fbdb-4a70-9b5d-5d827d41da23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:50.368365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:50.369709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:50.370645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:50.370645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:50.370645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:50.370645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 522/1069] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:50.371642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.375172 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe616f9-fbdb-4a70-9b5d-5d827d41da23 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.394902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ceb48-0b24-4498-884f-66f2420edc4a None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.434629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dddeb-9b30-4ab0-b4c5-80f5793f1d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.435302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dddeb-9b30-4ab0-b4c5-80f5793f1d1f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.435520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dddeb-9b30-4ab0-b4c5-80f5793f1d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.435797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dddeb-9b30-4ab0-b4c5-80f5793f1d1f None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.436467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dddeb-9b30-4ab0-b4c5-80f5793f1d1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:50.436772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dddeb-9b30-4ab0-b4c5-80f5793f1d1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.438312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dddeb-9b30-4ab0-b4c5-80f5793f1d1f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.443541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.447099 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 548/1070] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:50.452941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dddeb-9b30-4ab0-b4c5-80f5793f1d1f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.485024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe616f9-fbdb-4a70-9b5d-5d827d41da23 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.522747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642931a-3aba-4714-92e5-23ce80a0a82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.523358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642931a-3aba-4714-92e5-23ce80a0a82f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.523638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642931a-3aba-4714-92e5-23ce80a0a82f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.523912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642931a-3aba-4714-92e5-23ce80a0a82f None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.524551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642931a-3aba-4714-92e5-23ce80a0a82f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:50.524858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642931a-3aba-4714-92e5-23ce80a0a82f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.526217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642931a-3aba-4714-92e5-23ce80a0a82f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.538997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 523/1071] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:50.544403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642931a-3aba-4714-92e5-23ce80a0a82f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.566412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dddeb-9b30-4ab0-b4c5-80f5793f1d1f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.601477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20765b2e-b90e-4b8a-ad6a-8024ff1ad960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.602223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20765b2e-b90e-4b8a-ad6a-8024ff1ad960 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.602578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20765b2e-b90e-4b8a-ad6a-8024ff1ad960 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.602908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20765b2e-b90e-4b8a-ad6a-8024ff1ad960 None admin] PATH_INFO: `/v3/users/79ed6a0bf1364c269e43ae7e83dfec89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.604059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20765b2e-b90e-4b8a-ad6a-8024ff1ad960 None admin] RBAC: Authorizing `identity:get_user(user_id=79ed6a0bf1364c269e43ae7e83dfec89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.605492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20765b2e-b90e-4b8a-ad6a-8024ff1ad960 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.607107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 549/1072] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:31:50 2022] GET /identity/v3/users/79ed6a0bf1364c269e43ae7e83dfec89 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:50.607866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20765b2e-b90e-4b8a-ad6a-8024ff1ad960 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.627287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.651170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642931a-3aba-4714-92e5-23ce80a0a82f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.676242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.689023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228586c8-4d44-4eda-9154-1e357c6dfd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.689655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228586c8-4d44-4eda-9154-1e357c6dfd00 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.689938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228586c8-4d44-4eda-9154-1e357c6dfd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.690219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228586c8-4d44-4eda-9154-1e357c6dfd00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.697474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228586c8-4d44-4eda-9154-1e357c6dfd00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.698960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228586c8-4d44-4eda-9154-1e357c6dfd00 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-228586c8-4d44-4eda-9154-1e357c6dfd00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:50.711831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:50.713009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:50.714032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:50.717680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 524/1073] 10.222.0.82 () {56 vars in 1093 bytes} [Wed Aug 24 03:31:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:50.718710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228586c8-4d44-4eda-9154-1e357c6dfd00 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.727046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20765b2e-b90e-4b8a-ad6a-8024ff1ad960 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.760414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c702717-ad70-49e8-859d-a4cbaadb34e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.761027 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c702717-ad70-49e8-859d-a4cbaadb34e5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.761310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c702717-ad70-49e8-859d-a4cbaadb34e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.761594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c702717-ad70-49e8-859d-a4cbaadb34e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.822400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228586c8-4d44-4eda-9154-1e357c6dfd00 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:50.834292 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c702717-ad70-49e8-859d-a4cbaadb34e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.835840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c702717-ad70-49e8-859d-a4cbaadb34e5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.846348 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.870746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dc96b-f6ac-4adb-9cba-74d8f0732116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:50.871397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dc96b-f6ac-4adb-9cba-74d8f0732116 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:50.871679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dc96b-f6ac-4adb-9cba-74d8f0732116 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:50.871956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dc96b-f6ac-4adb-9cba-74d8f0732116 None admin] PATH_INFO: `/v3/projects/a405a5ff63284c4ebc649d18a620ff38` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:50.873103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156dc96b-f6ac-4adb-9cba-74d8f0732116 None admin] RBAC: Authorizing `identity:get_project(project_id=a405a5ff63284c4ebc649d18a620ff38)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:50.874589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156dc96b-f6ac-4adb-9cba-74d8f0732116 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:50.877020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 525/1074] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:31:50 2022] GET /identity/v3/projects/a405a5ff63284c4ebc649d18a620ff38 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:50.877815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dc96b-f6ac-4adb-9cba-74d8f0732116 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.935297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:50.939672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 550/1075] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:31:50.940744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c702717-ad70-49e8-859d-a4cbaadb34e5 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:50.990977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dc96b-f6ac-4adb-9cba-74d8f0732116 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.027110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a838d3-f1e7-4ba3-87e2-cee54163db2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.027723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a838d3-f1e7-4ba3-87e2-cee54163db2e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.028034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a838d3-f1e7-4ba3-87e2-cee54163db2e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.028316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a838d3-f1e7-4ba3-87e2-cee54163db2e None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.029057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a838d3-f1e7-4ba3-87e2-cee54163db2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:51.029369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a838d3-f1e7-4ba3-87e2-cee54163db2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.030908 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a838d3-f1e7-4ba3-87e2-cee54163db2e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.047471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c702717-ad70-49e8-859d-a4cbaadb34e5 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.049856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 526/1076] 10.222.0.82 () {56 vars in 1133 bytes} [Wed Aug 24 03:31:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:51.050758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a838d3-f1e7-4ba3-87e2-cee54163db2e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.057394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:51.083635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b946a3-20d9-4b4a-aff5-158d03fc2e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.084317 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b946a3-20d9-4b4a-aff5-158d03fc2e40 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.084601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b946a3-20d9-4b4a-aff5-158d03fc2e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.084876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b946a3-20d9-4b4a-aff5-158d03fc2e40 None admin] PATH_INFO: `/v3/projects/a405a5ff63284c4ebc649d18a620ff38/users/79ed6a0bf1364c269e43ae7e83dfec89/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.087077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b946a3-20d9-4b4a-aff5-158d03fc2e40 None admin] RBAC: Authorizing `identity:create_grant(project_id=a405a5ff63284c4ebc649d18a620ff38, user_id=79ed6a0bf1364c269e43ae7e83dfec89, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.088923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b946a3-20d9-4b4a-aff5-158d03fc2e40 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.158746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a838d3-f1e7-4ba3-87e2-cee54163db2e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.170005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:51.200130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617ab29-4013-48f5-84a4-4aacc8b7cd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.200781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617ab29-4013-48f5-84a4-4aacc8b7cd2b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.201057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617ab29-4013-48f5-84a4-4aacc8b7cd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.201328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617ab29-4013-48f5-84a4-4aacc8b7cd2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.208746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 551/1077] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:31:50 2022] PUT /identity/v3/projects/a405a5ff63284c4ebc649d18a620ff38/users/79ed6a0bf1364c269e43ae7e83dfec89/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:51.214136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b946a3-20d9-4b4a-aff5-158d03fc2e40 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.214552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617ab29-4013-48f5-84a4-4aacc8b7cd2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.215963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617ab29-4013-48f5-84a4-4aacc8b7cd2b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-c617ab29-4013-48f5-84a4-4aacc8b7cd2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:51.235602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:51.237589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:51.240762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:51.240762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:51.240762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:51.240762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 527/1078] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:51.244182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617ab29-4013-48f5-84a4-4aacc8b7cd2b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.369882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617ab29-4013-48f5-84a4-4aacc8b7cd2b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.376609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b946a3-20d9-4b4a-aff5-158d03fc2e40 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.414346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c037c-701f-487b-a2bf-6a9debf94fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.415357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c037c-701f-487b-a2bf-6a9debf94fcd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.415794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c037c-701f-487b-a2bf-6a9debf94fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.415910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5563ba-ccb2-48b0-880e-4b1317ef0bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.416330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c037c-701f-487b-a2bf-6a9debf94fcd None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.416563 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5563ba-ccb2-48b0-880e-4b1317ef0bbe None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.416849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5563ba-ccb2-48b0-880e-4b1317ef0bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.417129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5563ba-ccb2-48b0-880e-4b1317ef0bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.417601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c037c-701f-487b-a2bf-6a9debf94fcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:51.418056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c037c-701f-487b-a2bf-6a9debf94fcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.420232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c037c-701f-487b-a2bf-6a9debf94fcd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.428683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5563ba-ccb2-48b0-880e-4b1317ef0bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.430148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5563ba-ccb2-48b0-880e-4b1317ef0bbe None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.437019 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 528/1079] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-1f5563ba-ccb2-48b0-880e-4b1317ef0bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:51.451665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:51.454385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:51.456123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:51.456123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:51.456123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:51.456123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 552/1080] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:51.460656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c037c-701f-487b-a2bf-6a9debf94fcd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.494327 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:51.586731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c037c-701f-487b-a2bf-6a9debf94fcd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.632869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ec912-7268-4a5b-9618-c488423cf6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.633908 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ec912-7268-4a5b-9618-c488423cf6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.634330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ec912-7268-4a5b-9618-c488423cf6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.634787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ec912-7268-4a5b-9618-c488423cf6f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.635707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ec912-7268-4a5b-9618-c488423cf6f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:51.636261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ec912-7268-4a5b-9618-c488423cf6f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.637995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ec912-7268-4a5b-9618-c488423cf6f6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.650929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 529/1081] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:51.657357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5563ba-ccb2-48b0-880e-4b1317ef0bbe None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.684291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:51.772206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5563ba-ccb2-48b0-880e-4b1317ef0bbe None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.806427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980c0f4-8bfa-424a-b419-2ffd12cff621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.807080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980c0f4-8bfa-424a-b419-2ffd12cff621 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.807357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980c0f4-8bfa-424a-b419-2ffd12cff621 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.807629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980c0f4-8bfa-424a-b419-2ffd12cff621 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.815021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980c0f4-8bfa-424a-b419-2ffd12cff621 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.816458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980c0f4-8bfa-424a-b419-2ffd12cff621 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-b980c0f4-8bfa-424a-b419-2ffd12cff621 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:51.827860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:51.830029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:51.831072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:51.831072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:51.831072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:51.831072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:51.831072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:51.831072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:51.831072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:51.831072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:51.831072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 553/1082] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:51.835761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ec912-7268-4a5b-9618-c488423cf6f6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:51.927990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:51.946019 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ec912-7268-4a5b-9618-c488423cf6f6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:51.985019 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bdf33-a704-4063-877a-e944c9bc6921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:51.985630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bdf33-a704-4063-877a-e944c9bc6921 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:51.985911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bdf33-a704-4063-877a-e944c9bc6921 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:51.986190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bdf33-a704-4063-877a-e944c9bc6921 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:51.986926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bdf33-a704-4063-877a-e944c9bc6921 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:51.987232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bdf33-a704-4063-877a-e944c9bc6921 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:51.988790 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bdf33-a704-4063-877a-e944c9bc6921 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:52.003344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 530/1083] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:52.008051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980c0f4-8bfa-424a-b419-2ffd12cff621 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:52.093281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:52.112705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980c0f4-8bfa-424a-b419-2ffd12cff621 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:52.147082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce094a5f-064b-41f2-95cd-c8e8baf7d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:52.147732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094a5f-064b-41f2-95cd-c8e8baf7d888 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:52.148009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094a5f-064b-41f2-95cd-c8e8baf7d888 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:52.148276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094a5f-064b-41f2-95cd-c8e8baf7d888 None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/0f34fe3fe912498b90dbf950e7fc999d/roles/c29a8da689b642ffb03ab69775d7c627` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:52.150433 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce094a5f-064b-41f2-95cd-c8e8baf7d888 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=0f34fe3fe912498b90dbf950e7fc999d, role_id=c29a8da689b642ffb03ab69775d7c627)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:52.151896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce094a5f-064b-41f2-95cd-c8e8baf7d888 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:52.225320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 554/1084] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:31:52 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/0f34fe3fe912498b90dbf950e7fc999d/roles/c29a8da689b642ffb03ab69775d7c627 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:31:53.155239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425055-09b1-45aa-9ad7-e10f7993f061 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:53.155791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425055-09b1-45aa-9ad7-e10f7993f061 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:53.155791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425055-09b1-45aa-9ad7-e10f7993f061 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:53.157333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 531/1085] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:53.162594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f341ec-d241-483b-ba98-c80d93583680 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:53.163005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f341ec-d241-483b-ba98-c80d93583680 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:53.163158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f341ec-d241-483b-ba98-c80d93583680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:53.181760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-90f341ec-d241-483b-ba98-c80d93583680 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:53.182983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-90f341ec-d241-483b-ba98-c80d93583680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:53.254220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 555/1086] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:53.528568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29425055-09b1-45aa-9ad7-e10f7993f061 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:53.637088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29425055-09b1-45aa-9ad7-e10f7993f061 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:53.670403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db585610-3194-4a3b-b776-dc376f5612e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:53.671071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db585610-3194-4a3b-b776-dc376f5612e9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:53.671352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db585610-3194-4a3b-b776-dc376f5612e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:53.671625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db585610-3194-4a3b-b776-dc376f5612e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:53.680066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:53.683578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db585610-3194-4a3b-b776-dc376f5612e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-db585610-3194-4a3b-b776-dc376f5612e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:53.689318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:53.690554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:53.691451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 532/1087] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:31:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:53.698118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f341ec-d241-483b-ba98-c80d93583680 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:53.770584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:53.806009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f341ec-d241-483b-ba98-c80d93583680 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:53.842535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2cc5-ef28-431c-bcdb-8c6d7f0a87c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:53.843149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2cc5-ef28-431c-bcdb-8c6d7f0a87c4 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:53.843428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2cc5-ef28-431c-bcdb-8c6d7f0a87c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:53.843696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2cc5-ef28-431c-bcdb-8c6d7f0a87c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:53.845031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2cc5-ef28-431c-bcdb-8c6d7f0a87c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:53.845392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2cc5-ef28-431c-bcdb-8c6d7f0a87c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:53.847126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2cc5-ef28-431c-bcdb-8c6d7f0a87c4 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:53.856155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 556/1088] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:31:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:53.863105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db585610-3194-4a3b-b776-dc376f5612e9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:53.967555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db585610-3194-4a3b-b776-dc376f5612e9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:54.004577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8913a8-cced-4992-a04a-46d2ff40282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:54.005258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8913a8-cced-4992-a04a-46d2ff40282e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:54.005844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8913a8-cced-4992-a04a-46d2ff40282e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:54.006211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8913a8-cced-4992-a04a-46d2ff40282e None admin] PATH_INFO: `/v3/users/79ed6a0bf1364c269e43ae7e83dfec89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:54.008128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8913a8-cced-4992-a04a-46d2ff40282e None admin] RBAC: Authorizing `identity:get_user(user_id=79ed6a0bf1364c269e43ae7e83dfec89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:54.010187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8913a8-cced-4992-a04a-46d2ff40282e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:54.016978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 533/1089] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:31:53 2022] GET /identity/v3/users/79ed6a0bf1364c269e43ae7e83dfec89 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:54.017917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2cc5-ef28-431c-bcdb-8c6d7f0a87c4 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:54.145019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2cc5-ef28-431c-bcdb-8c6d7f0a87c4 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:54.181113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:54.186322 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d6c5b-b2a1-441d-8f25-a3fdf340382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:54.186992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d6c5b-b2a1-441d-8f25-a3fdf340382f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:54.187268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d6c5b-b2a1-441d-8f25-a3fdf340382f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:54.187541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d6c5b-b2a1-441d-8f25-a3fdf340382f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:54.251309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e068dee-678d-49c3-94bc-dc507ba76bad None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:54.251906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e068dee-678d-49c3-94bc-dc507ba76bad None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:54.251906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e068dee-678d-49c3-94bc-dc507ba76bad None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:54.253195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 534/1090] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:54.258219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a2665-503c-4abe-ac4d-f2906ed2d2cb None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:54.258534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a2665-503c-4abe-ac4d-f2906ed2d2cb None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:54.258838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a2665-503c-4abe-ac4d-f2906ed2d2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:54.261651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d6c5b-b2a1-441d-8f25-a3fdf340382f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:54.263138 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d6c5b-b2a1-441d-8f25-a3fdf340382f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:54.278638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-097a2665-503c-4abe-ac4d-f2906ed2d2cb None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:54.279878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-097a2665-503c-4abe-ac4d-f2906ed2d2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:54.351022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:54.356297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 535/1091] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:54.375353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 557/1092] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:31:54.377082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a2665-503c-4abe-ac4d-f2906ed2d2cb None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:54.483474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a2665-503c-4abe-ac4d-f2906ed2d2cb None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:54.525835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2ecd0-b412-400a-aa3d-e084ff265a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:54.526588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2ecd0-b412-400a-aa3d-e084ff265a48 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:54.527100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2ecd0-b412-400a-aa3d-e084ff265a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:54.527541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2ecd0-b412-400a-aa3d-e084ff265a48 None admin] PATH_INFO: `/v3/projects/a405a5ff63284c4ebc649d18a620ff38` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:54.528813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2ecd0-b412-400a-aa3d-e084ff265a48 None admin] RBAC: Authorizing `identity:get_project(project_id=a405a5ff63284c4ebc649d18a620ff38)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:54.530317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2ecd0-b412-400a-aa3d-e084ff265a48 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:54.532679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 536/1093] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:31:54 2022] GET /identity/v3/projects/a405a5ff63284c4ebc649d18a620ff38 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:54.539349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d6c5b-b2a1-441d-8f25-a3fdf340382f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:54.570224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:54.638747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2ecd0-b412-400a-aa3d-e084ff265a48 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:54.641928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d6c5b-b2a1-441d-8f25-a3fdf340382f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:54.675453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21c91c-ce37-4c0b-b641-7cf955f1a826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:54.676076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21c91c-ce37-4c0b-b641-7cf955f1a826 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:54.676354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21c91c-ce37-4c0b-b641-7cf955f1a826 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:54.676624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21c91c-ce37-4c0b-b641-7cf955f1a826 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:54.677356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21c91c-ce37-4c0b-b641-7cf955f1a826 None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=a405a5ff63284c4ebc649d18a620ff38, user.id=79ed6a0bf1364c269e43ae7e83dfec89) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:54.677665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21c91c-ce37-4c0b-b641-7cf955f1a826 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:54.679398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21c91c-ce37-4c0b-b641-7cf955f1a826 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:54.687733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 558/1094] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:54 2022] GET /identity/v3/role_assignments?user.id=79ed6a0bf1364c269e43ae7e83dfec89&scope.project.id=a405a5ff63284c4ebc649d18a620ff38&role.id=9e904216086e4a06abf9216b0c863f89 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:54.714265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:54.758544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2ecd0-b412-400a-aa3d-e084ff265a48 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:54.796917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a340c-3b1a-4fa2-b89d-7d1fc4cf2719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:54.797652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a340c-3b1a-4fa2-b89d-7d1fc4cf2719 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:54.798280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a340c-3b1a-4fa2-b89d-7d1fc4cf2719 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:54.798280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a340c-3b1a-4fa2-b89d-7d1fc4cf2719 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:54.809553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a340c-3b1a-4fa2-b89d-7d1fc4cf2719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-b28a340c-3b1a-4fa2-b89d-7d1fc4cf2719 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:31:54.815304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:54.816539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:54.817414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 537/1095] 10.222.0.82 () {56 vars in 1120 bytes} [Wed Aug 24 03:31:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:54.822882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21c91c-ce37-4c0b-b641-7cf955f1a826 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:54.924408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:54.934903 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21c91c-ce37-4c0b-b641-7cf955f1a826 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:54.979684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ed9c9-be3d-424e-9198-a64d88d07e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:54.980308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ed9c9-be3d-424e-9198-a64d88d07e21 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:54.980600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ed9c9-be3d-424e-9198-a64d88d07e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:54.980877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ed9c9-be3d-424e-9198-a64d88d07e21 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:54.981564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ed9c9-be3d-424e-9198-a64d88d07e21 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:54.981875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ed9c9-be3d-424e-9198-a64d88d07e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:54.983296 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ed9c9-be3d-424e-9198-a64d88d07e21 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:54.993010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 559/1096] 10.222.0.82 () {56 vars in 1115 bytes} [Wed Aug 24 03:31:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:54.999080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a340c-3b1a-4fa2-b89d-7d1fc4cf2719 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:55.112616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a340c-3b1a-4fa2-b89d-7d1fc4cf2719 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:55.142084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:55.152245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d09cd-6753-43da-a864-3dc67924217d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:55.152881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d09cd-6753-43da-a864-3dc67924217d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:55.153165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d09cd-6753-43da-a864-3dc67924217d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:55.153438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d09cd-6753-43da-a864-3dc67924217d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:55.163845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d09cd-6753-43da-a864-3dc67924217d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:55.165320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d09cd-6753-43da-a864-3dc67924217d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-795d09cd-6753-43da-a864-3dc67924217d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:55.180830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:55.181909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:55.182876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:55.182876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:55.182876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:55.182876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 538/1097] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:31:55.187996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ed9c9-be3d-424e-9198-a64d88d07e21 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:55.297021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ed9c9-be3d-424e-9198-a64d88d07e21 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:55.330689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8992f-819b-480b-8647-e1b3381b8119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:55.331299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8992f-819b-480b-8647-e1b3381b8119 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:55.331579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8992f-819b-480b-8647-e1b3381b8119 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:55.331850 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8992f-819b-480b-8647-e1b3381b8119 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:55.332488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8992f-819b-480b-8647-e1b3381b8119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:55.332791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8992f-819b-480b-8647-e1b3381b8119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:55.334159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8992f-819b-480b-8647-e1b3381b8119 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:55.346941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 560/1098] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:55.352642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d09cd-6753-43da-a864-3dc67924217d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:55.455395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d09cd-6753-43da-a864-3dc67924217d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:55.492954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3ad2f-53e2-4de9-969a-d0c44777a11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:55.493582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ad2f-53e2-4de9-969a-d0c44777a11c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:55.493867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ad2f-53e2-4de9-969a-d0c44777a11c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:55.494143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ad2f-53e2-4de9-969a-d0c44777a11c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:55.501700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3ad2f-53e2-4de9-969a-d0c44777a11c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:55.503201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3ad2f-53e2-4de9-969a-d0c44777a11c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-71b3ad2f-53e2-4de9-969a-d0c44777a11c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:31:55.514193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:55.515250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:31:55.516212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 539/1099] 10.222.0.82 () {56 vars in 1093 bytes} [Wed Aug 24 03:31:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:55.521202 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8992f-819b-480b-8647-e1b3381b8119 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:55.541551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:55.629419 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8992f-819b-480b-8647-e1b3381b8119 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:55.666732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4e18f-cafd-435b-ad1b-6e4a47559574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:55.667346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4e18f-cafd-435b-ad1b-6e4a47559574 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:55.667631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4e18f-cafd-435b-ad1b-6e4a47559574 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:55.667907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4e18f-cafd-435b-ad1b-6e4a47559574 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:55.668643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4e18f-cafd-435b-ad1b-6e4a47559574 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:55.668957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4e18f-cafd-435b-ad1b-6e4a47559574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:55.670391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4e18f-cafd-435b-ad1b-6e4a47559574 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:55.689147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 561/1100] 10.222.0.82 () {56 vars in 1133 bytes} [Wed Aug 24 03:31:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:55.695546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3ad2f-53e2-4de9-969a-d0c44777a11c None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:55.736175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:55.805923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3ad2f-53e2-4de9-969a-d0c44777a11c None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:55.847081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77474f-3ee5-40f8-b0f5-edb451d743f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:55.847684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77474f-3ee5-40f8-b0f5-edb451d743f7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:55.847963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77474f-3ee5-40f8-b0f5-edb451d743f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:55.848235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77474f-3ee5-40f8-b0f5-edb451d743f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:55.925227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77474f-3ee5-40f8-b0f5-edb451d743f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:55.926761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77474f-3ee5-40f8-b0f5-edb451d743f7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:55.968074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:56.045655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 540/1101] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:31:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:31:56.052246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4e18f-cafd-435b-ad1b-6e4a47559574 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:56.135394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:56.159380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4e18f-cafd-435b-ad1b-6e4a47559574 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:56.197150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8d633-79c6-4cf2-aa1b-b83c301d7581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:56.197782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8d633-79c6-4cf2-aa1b-b83c301d7581 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:56.197959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8d633-79c6-4cf2-aa1b-b83c301d7581 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:56.198226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8d633-79c6-4cf2-aa1b-b83c301d7581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:56.209380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8d633-79c6-4cf2-aa1b-b83c301d7581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:56.210869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8d633-79c6-4cf2-aa1b-b83c301d7581 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-fff8d633-79c6-4cf2-aa1b-b83c301d7581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:31:56.225246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:56.226293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:31:56.227272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:31:56.227272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:31:56.227272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:56.227272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 562/1102] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:56.233297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77474f-3ee5-40f8-b0f5-edb451d743f7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:56.340796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77474f-3ee5-40f8-b0f5-edb451d743f7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:56.378001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1379d-80fd-467a-90bc-afadecfb2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:56.378663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1379d-80fd-467a-90bc-afadecfb2bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:56.378919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1379d-80fd-467a-90bc-afadecfb2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:56.379194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1379d-80fd-467a-90bc-afadecfb2bf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:56.379844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1379d-80fd-467a-90bc-afadecfb2bf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:56.380151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1379d-80fd-467a-90bc-afadecfb2bf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:56.381536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1379d-80fd-467a-90bc-afadecfb2bf2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:56.388606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:56.394474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 541/1103] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:56.400411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8d633-79c6-4cf2-aa1b-b83c301d7581 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:56.501972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8d633-79c6-4cf2-aa1b-b83c301d7581 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:56.527729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:56.535442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0649c-e746-4992-9ec0-6e2a0beb12dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:56.536081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0649c-e746-4992-9ec0-6e2a0beb12dc None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:56.536363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0649c-e746-4992-9ec0-6e2a0beb12dc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:56.536684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0649c-e746-4992-9ec0-6e2a0beb12dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:56.544206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0649c-e746-4992-9ec0-6e2a0beb12dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:56.545944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0649c-e746-4992-9ec0-6e2a0beb12dc None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-33f0649c-e746-4992-9ec0-6e2a0beb12dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:31:56.558423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:31:56.559399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:31:56.560546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:31:56.560546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:31:56.560546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:31:56.560546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:31:56.560546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:31:56.560546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:31:56.560546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:31:56.560546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:31:56.560546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 563/1104] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:31:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:31:56.566312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1379d-80fd-467a-90bc-afadecfb2bf2 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:56.671136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1379d-80fd-467a-90bc-afadecfb2bf2 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:56.680393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d39966-bd9f-4132-a788-91fb89fe5842 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:56.680789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d39966-bd9f-4132-a788-91fb89fe5842 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:56.681072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d39966-bd9f-4132-a788-91fb89fe5842 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:56.682414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 564/1105] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:31:56.687328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29904847-f582-4367-9144-ab952e914896 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:56.687615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29904847-f582-4367-9144-ab952e914896 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:56.687897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29904847-f582-4367-9144-ab952e914896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:56.708200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-29904847-f582-4367-9144-ab952e914896 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:56.709667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aee294-c422-4b51-83d1-ffeec376b5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:56.709851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-29904847-f582-4367-9144-ab952e914896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:56.710285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aee294-c422-4b51-83d1-ffeec376b5ab None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:56.710593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aee294-c422-4b51-83d1-ffeec376b5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:56.710872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aee294-c422-4b51-83d1-ffeec376b5ab None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:56.711575 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aee294-c422-4b51-83d1-ffeec376b5ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:56.711884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aee294-c422-4b51-83d1-ffeec376b5ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:56.713337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aee294-c422-4b51-83d1-ffeec376b5ab None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:56.727795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 542/1106] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:31:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:56.733982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aee294-c422-4b51-83d1-ffeec376b5ab None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:56.790729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 565/1107] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:31:56.803958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:56.849235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aee294-c422-4b51-83d1-ffeec376b5ab None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:56.883016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebaf64d-fb8d-43f0-9595-4f52981230ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:56.883621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaf64d-fb8d-43f0-9595-4f52981230ff None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:56.883901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaf64d-fb8d-43f0-9595-4f52981230ff None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:56.884172 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaf64d-fb8d-43f0-9595-4f52981230ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:56.884894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebaf64d-fb8d-43f0-9595-4f52981230ff None admin] RBAC: Adding query filter params (role.id=c29a8da689b642ffb03ab69775d7c627, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=0f34fe3fe912498b90dbf950e7fc999d) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:31:56.885200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebaf64d-fb8d-43f0-9595-4f52981230ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:56.886689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebaf64d-fb8d-43f0-9595-4f52981230ff None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:56.892800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 543/1108] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:31:56 2022] GET /identity/v3/role_assignments?user.id=0f34fe3fe912498b90dbf950e7fc999d&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=c29a8da689b642ffb03ab69775d7c627 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:31:57.085345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29904847-f582-4367-9144-ab952e914896 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:57.190828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29904847-f582-4367-9144-ab952e914896 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:57.212529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:57.231650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a2e08-ee03-4c53-b093-23b85c06e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:57.232291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a2e08-ee03-4c53-b093-23b85c06e745 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:57.232590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a2e08-ee03-4c53-b093-23b85c06e745 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:57.233016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a2e08-ee03-4c53-b093-23b85c06e745 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:57.234132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a2e08-ee03-4c53-b093-23b85c06e745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:57.235926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a2e08-ee03-4c53-b093-23b85c06e745 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:57.238412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 566/1109] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:31:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:31:57.244705 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebaf64d-fb8d-43f0-9595-4f52981230ff None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:57.361331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebaf64d-fb8d-43f0-9595-4f52981230ff None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:57.384667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:57.398098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1b985-5c58-451d-a700-7e35dd33b519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:57.398752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1b985-5c58-451d-a700-7e35dd33b519 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:57.399201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1b985-5c58-451d-a700-7e35dd33b519 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:57.399672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1b985-5c58-451d-a700-7e35dd33b519 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:57.400379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1b985-5c58-451d-a700-7e35dd33b519 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:57.401824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1b985-5c58-451d-a700-7e35dd33b519 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:57.518359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 544/1110] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:31:57 2022] POST /identity/v3/users => generated 286 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:31:59.557893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540acea9-4a09-4619-b916-b4d1cef1e3c8 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:59.558424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540acea9-4a09-4619-b916-b4d1cef1e3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:59.558424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540acea9-4a09-4619-b916-b4d1cef1e3c8 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:59.559809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 567/1111] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:59.565591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ace93-78b3-4a90-8656-df02a9cb4ad1 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:59.565996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ace93-78b3-4a90-8656-df02a9cb4ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:59.566144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ace93-78b3-4a90-8656-df02a9cb4ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:59.584360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-bb3ace93-78b3-4a90-8656-df02a9cb4ad1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:59.585560 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bb3ace93-78b3-4a90-8656-df02a9cb4ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:59.653635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 545/1112] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:31:59.924420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540acea9-4a09-4619-b916-b4d1cef1e3c8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:00.037350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540acea9-4a09-4619-b916-b4d1cef1e3c8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:00.078175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed7d65-7585-4244-8f98-e1494fcac4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:00.083447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7d65-7585-4244-8f98-e1494fcac4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:00.083447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7d65-7585-4244-8f98-e1494fcac4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:00.083447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7d65-7585-4244-8f98-e1494fcac4e1 None admin] PATH_INFO: `/v3/roles/e5de7bbccf114283851bcc0028d7cc51` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:00.083447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed7d65-7585-4244-8f98-e1494fcac4e1 None admin] RBAC: Authorizing `identity:get_role(role_id=e5de7bbccf114283851bcc0028d7cc51)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:00.085103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed7d65-7585-4244-8f98-e1494fcac4e1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:00.086558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 568/1113] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:31:59 2022] GET /identity/v3/roles/e5de7bbccf114283851bcc0028d7cc51 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:00.092483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ace93-78b3-4a90-8656-df02a9cb4ad1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:00.195736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ace93-78b3-4a90-8656-df02a9cb4ad1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:00.231643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:00.259573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4453f-3932-4f3f-86f8-fc9b8d64ebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:00.260210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4453f-3932-4f3f-86f8-fc9b8d64ebfd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:00.260493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4453f-3932-4f3f-86f8-fc9b8d64ebfd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:00.260760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4453f-3932-4f3f-86f8-fc9b8d64ebfd None admin] PATH_INFO: `/v3/users/33b1191133084c4689425c127c7b09bd` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:00.274380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4453f-3932-4f3f-86f8-fc9b8d64ebfd None admin] RBAC: Authorizing `identity:get_user(user_id=33b1191133084c4689425c127c7b09bd)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:00.275875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4453f-3932-4f3f-86f8-fc9b8d64ebfd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:00.277501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 546/1114] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:00 2022] GET /identity/v3/users/33b1191133084c4689425c127c7b09bd => generated 286 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:00.283733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed7d65-7585-4244-8f98-e1494fcac4e1 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:00.341452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:00.387240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed7d65-7585-4244-8f98-e1494fcac4e1 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:00.422145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9299b-c61f-4119-b188-b98a2ecc297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:00.423441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9299b-c61f-4119-b188-b98a2ecc297f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:00.423859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9299b-c61f-4119-b188-b98a2ecc297f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:00.424224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9299b-c61f-4119-b188-b98a2ecc297f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:00.493309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9299b-c61f-4119-b188-b98a2ecc297f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:00.494764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9299b-c61f-4119-b188-b98a2ecc297f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:00.547909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:00.592564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 569/1115] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:32:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:32:00.599441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4453f-3932-4f3f-86f8-fc9b8d64ebfd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:00.700927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4453f-3932-4f3f-86f8-fc9b8d64ebfd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:00.739126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d7efa-04a9-4a56-82d9-e3b9ad59e316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:00.739757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d7efa-04a9-4a56-82d9-e3b9ad59e316 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:00.740292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d7efa-04a9-4a56-82d9-e3b9ad59e316 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:00.740571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d7efa-04a9-4a56-82d9-e3b9ad59e316 None admin] PATH_INFO: `/v3/projects/a405a5ff63284c4ebc649d18a620ff38` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:00.741695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d7efa-04a9-4a56-82d9-e3b9ad59e316 None admin] RBAC: Authorizing `identity:get_project(project_id=a405a5ff63284c4ebc649d18a620ff38)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:00.743182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d7efa-04a9-4a56-82d9-e3b9ad59e316 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:00.744867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 547/1116] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:32:00 2022] GET /identity/v3/projects/a405a5ff63284c4ebc649d18a620ff38 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:00.752666 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9299b-c61f-4119-b188-b98a2ecc297f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:00.854341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9299b-c61f-4119-b188-b98a2ecc297f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:00.888010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e64377-5d58-4bbb-a0c7-68380be19758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:00.888663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e64377-5d58-4bbb-a0c7-68380be19758 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:00.899135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e64377-5d58-4bbb-a0c7-68380be19758 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:00.899135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e64377-5d58-4bbb-a0c7-68380be19758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:00.899135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e64377-5d58-4bbb-a0c7-68380be19758 None admin] RBAC: Adding query filter params (role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=a405a5ff63284c4ebc649d18a620ff38, user.id=33b1191133084c4689425c127c7b09bd) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:00.899135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e64377-5d58-4bbb-a0c7-68380be19758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:00.899135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e64377-5d58-4bbb-a0c7-68380be19758 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:00.899599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 570/1117] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:32:00 2022] GET /identity/v3/role_assignments?user.id=33b1191133084c4689425c127c7b09bd&scope.project.id=a405a5ff63284c4ebc649d18a620ff38&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:02.807632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9f89b-d001-4e47-ae14-3fdda8d73271 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:02.808171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9f89b-d001-4e47-ae14-3fdda8d73271 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:02.808171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9f89b-d001-4e47-ae14-3fdda8d73271 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:02.809492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 548/1118] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:02.815250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f1435-651b-491c-8b2e-cb69560b4091 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:02.815671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f1435-651b-491c-8b2e-cb69560b4091 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:02.815875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f1435-651b-491c-8b2e-cb69560b4091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:02.835484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-0f8f1435-651b-491c-8b2e-cb69560b4091 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:02.836705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0f8f1435-651b-491c-8b2e-cb69560b4091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:02.849901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:02.907260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 571/1119] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:03.178718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9f89b-d001-4e47-ae14-3fdda8d73271 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:03.214996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:03.283298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9f89b-d001-4e47-ae14-3fdda8d73271 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:03.318932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7ac4e-b08b-43b2-a0a3-a5b8c1af6f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:03.319565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7ac4e-b08b-43b2-a0a3-a5b8c1af6f35 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:03.319846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7ac4e-b08b-43b2-a0a3-a5b8c1af6f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:03.320117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7ac4e-b08b-43b2-a0a3-a5b8c1af6f35 None admin] PATH_INFO: `/v3/roles/e5de7bbccf114283851bcc0028d7cc51` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:03.321523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7ac4e-b08b-43b2-a0a3-a5b8c1af6f35 None admin] RBAC: Authorizing `identity:get_role(role_id=e5de7bbccf114283851bcc0028d7cc51)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:03.322939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7ac4e-b08b-43b2-a0a3-a5b8c1af6f35 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:03.324288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 549/1120] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:03 2022] GET /identity/v3/roles/e5de7bbccf114283851bcc0028d7cc51 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:03.330584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f1435-651b-491c-8b2e-cb69560b4091 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:03.443415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f1435-651b-491c-8b2e-cb69560b4091 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:03.479980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d8d6-ac9c-43c1-9b9b-9ce328e1cc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:03.480625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d8d6-ac9c-43c1-9b9b-9ce328e1cc97 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:03.480911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d8d6-ac9c-43c1-9b9b-9ce328e1cc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:03.481190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d8d6-ac9c-43c1-9b9b-9ce328e1cc97 None admin] PATH_INFO: `/v3/users/33b1191133084c4689425c127c7b09bd` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:03.482357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6d8d6-ac9c-43c1-9b9b-9ce328e1cc97 None admin] RBAC: Authorizing `identity:get_user(user_id=33b1191133084c4689425c127c7b09bd)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:03.483849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6d8d6-ac9c-43c1-9b9b-9ce328e1cc97 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:03.486579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 572/1121] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:03 2022] GET /identity/v3/users/33b1191133084c4689425c127c7b09bd => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:32:03.492851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7ac4e-b08b-43b2-a0a3-a5b8c1af6f35 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:03.599411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7ac4e-b08b-43b2-a0a3-a5b8c1af6f35 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:03.601133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:03.642736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fc607-1863-49ff-be5d-2c688c2a8fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:03.643384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fc607-1863-49ff-be5d-2c688c2a8fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:03.643674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fc607-1863-49ff-be5d-2c688c2a8fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:03.643958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fc607-1863-49ff-be5d-2c688c2a8fa4 None admin] PATH_INFO: `/v3/projects/a405a5ff63284c4ebc649d18a620ff38` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:03.645146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fc607-1863-49ff-be5d-2c688c2a8fa4 None admin] RBAC: Authorizing `identity:get_project(project_id=a405a5ff63284c4ebc649d18a620ff38)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:03.646733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fc607-1863-49ff-be5d-2c688c2a8fa4 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:03.648899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 550/1122] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:32:03 2022] GET /identity/v3/projects/a405a5ff63284c4ebc649d18a620ff38 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:03.655056 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d8d6-ac9c-43c1-9b9b-9ce328e1cc97 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:03.675159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:03.764319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d8d6-ac9c-43c1-9b9b-9ce328e1cc97 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:03.798493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a4b96-ca8d-4f14-a262-75dfa5eb99f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:03.799175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4b96-ca8d-4f14-a262-75dfa5eb99f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:03.799454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4b96-ca8d-4f14-a262-75dfa5eb99f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:03.799726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4b96-ca8d-4f14-a262-75dfa5eb99f3 None admin] PATH_INFO: `/v3/projects/a405a5ff63284c4ebc649d18a620ff38/users/33b1191133084c4689425c127c7b09bd/roles/e5de7bbccf114283851bcc0028d7cc51` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:03.801877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a4b96-ca8d-4f14-a262-75dfa5eb99f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a405a5ff63284c4ebc649d18a620ff38, user_id=33b1191133084c4689425c127c7b09bd, role_id=e5de7bbccf114283851bcc0028d7cc51)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:03.803462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a4b96-ca8d-4f14-a262-75dfa5eb99f3 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:03.877625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 573/1123] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:32:03 2022] PUT /identity/v3/projects/a405a5ff63284c4ebc649d18a620ff38/users/33b1191133084c4689425c127c7b09bd/roles/e5de7bbccf114283851bcc0028d7cc51 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:32:05.805968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a2cc4-e19f-4662-99f8-57ddf4fedfaa None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:05.806575 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a2cc4-e19f-4662-99f8-57ddf4fedfaa None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:05.806575 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a2cc4-e19f-4662-99f8-57ddf4fedfaa None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:05.807886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 551/1124] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:05.814068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61990bac-8fa3-4021-82ac-489aee72cb69 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:05.814631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61990bac-8fa3-4021-82ac-489aee72cb69 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:05.814812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61990bac-8fa3-4021-82ac-489aee72cb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:05.834302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-61990bac-8fa3-4021-82ac-489aee72cb69 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:05.835562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-61990bac-8fa3-4021-82ac-489aee72cb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:05.903026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 574/1125] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:06.181668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a2cc4-e19f-4662-99f8-57ddf4fedfaa None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:06.288368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a2cc4-e19f-4662-99f8-57ddf4fedfaa None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:06.323399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1e77-6110-4952-b290-ce2ac29e5815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:06.324026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1e77-6110-4952-b290-ce2ac29e5815 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:06.324311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1e77-6110-4952-b290-ce2ac29e5815 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:06.324588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1e77-6110-4952-b290-ce2ac29e5815 None admin] PATH_INFO: `/v3/roles/e5de7bbccf114283851bcc0028d7cc51` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:06.325983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d1e77-6110-4952-b290-ce2ac29e5815 None admin] RBAC: Authorizing `identity:get_role(role_id=e5de7bbccf114283851bcc0028d7cc51)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:06.327385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d1e77-6110-4952-b290-ce2ac29e5815 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:06.328784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 552/1126] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:06 2022] GET /identity/v3/roles/e5de7bbccf114283851bcc0028d7cc51 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:06.335085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61990bac-8fa3-4021-82ac-489aee72cb69 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:06.343225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:06.437653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61990bac-8fa3-4021-82ac-489aee72cb69 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:06.472190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41d3b5-050d-4a95-b18d-395cbca2160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:06.472817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41d3b5-050d-4a95-b18d-395cbca2160b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:06.473094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41d3b5-050d-4a95-b18d-395cbca2160b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:06.473364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41d3b5-050d-4a95-b18d-395cbca2160b None admin] PATH_INFO: `/v3/users/33b1191133084c4689425c127c7b09bd` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:06.474551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41d3b5-050d-4a95-b18d-395cbca2160b None admin] RBAC: Authorizing `identity:get_user(user_id=33b1191133084c4689425c127c7b09bd)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:06.475986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41d3b5-050d-4a95-b18d-395cbca2160b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:06.477584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 575/1127] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:06 2022] GET /identity/v3/users/33b1191133084c4689425c127c7b09bd => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:06.483682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1e77-6110-4952-b290-ce2ac29e5815 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:06.531449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:06.591238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1e77-6110-4952-b290-ce2ac29e5815 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:06.630691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bfead-3522-4b6b-a020-bd57fac77e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:06.631304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bfead-3522-4b6b-a020-bd57fac77e76 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:06.631591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bfead-3522-4b6b-a020-bd57fac77e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:06.631869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bfead-3522-4b6b-a020-bd57fac77e76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:06.702998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bfead-3522-4b6b-a020-bd57fac77e76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:06.704599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bfead-3522-4b6b-a020-bd57fac77e76 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:06.746094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:06.807907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 553/1128] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:32:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:32:06.814883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41d3b5-050d-4a95-b18d-395cbca2160b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:06.891757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:06.915368 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41d3b5-050d-4a95-b18d-395cbca2160b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:06.955890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f705b-3e01-4383-8ba1-6686926521bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:06.956436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f705b-3e01-4383-8ba1-6686926521bb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:06.956716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f705b-3e01-4383-8ba1-6686926521bb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:06.956992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f705b-3e01-4383-8ba1-6686926521bb None admin] PATH_INFO: `/v3/projects/a405a5ff63284c4ebc649d18a620ff38` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:06.958161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f705b-3e01-4383-8ba1-6686926521bb None admin] RBAC: Authorizing `identity:get_project(project_id=a405a5ff63284c4ebc649d18a620ff38)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:06.959650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f705b-3e01-4383-8ba1-6686926521bb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:06.961308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 576/1129] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:32:06 2022] GET /identity/v3/projects/a405a5ff63284c4ebc649d18a620ff38 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:06.967575 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bfead-3522-4b6b-a020-bd57fac77e76 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:07.068567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bfead-3522-4b6b-a020-bd57fac77e76 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:07.102475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9bb7e-106a-4df5-b445-55f3e1771960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:07.103093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9bb7e-106a-4df5-b445-55f3e1771960 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:07.103430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9bb7e-106a-4df5-b445-55f3e1771960 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:07.103715 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9bb7e-106a-4df5-b445-55f3e1771960 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:07.104433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9bb7e-106a-4df5-b445-55f3e1771960 None admin] RBAC: Adding query filter params (role.id=e5de7bbccf114283851bcc0028d7cc51, scope.project.id=a405a5ff63284c4ebc649d18a620ff38, user.id=33b1191133084c4689425c127c7b09bd) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:07.104735 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9bb7e-106a-4df5-b445-55f3e1771960 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:07.106160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9bb7e-106a-4df5-b445-55f3e1771960 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:07.112543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 554/1130] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:32:06 2022] GET /identity/v3/role_assignments?user.id=33b1191133084c4689425c127c7b09bd&scope.project.id=a405a5ff63284c4ebc649d18a620ff38&role.id=e5de7bbccf114283851bcc0028d7cc51 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:09.074702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a30438-d221-4528-98b4-b9b275dd123c None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:09.075280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a30438-d221-4528-98b4-b9b275dd123c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:09.075280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a30438-d221-4528-98b4-b9b275dd123c None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:09.076606 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 577/1131] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:09.082853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4225-b4d0-4c96-93c2-f0843d431bc6 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:09.083305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4225-b4d0-4c96-93c2-f0843d431bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:09.083408 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4225-b4d0-4c96-93c2-f0843d431bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:09.089693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:09.102749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-bf8e4225-b4d0-4c96-93c2-f0843d431bc6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:09.104676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bf8e4225-b4d0-4c96-93c2-f0843d431bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:09.171386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 555/1132] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:09.443457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a30438-d221-4528-98b4-b9b275dd123c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:09.543839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a30438-d221-4528-98b4-b9b275dd123c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:09.577364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31393efc-5fda-4097-ad10-72b12ae565ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:09.577991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31393efc-5fda-4097-ad10-72b12ae565ea None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:09.578278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31393efc-5fda-4097-ad10-72b12ae565ea None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:09.578640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31393efc-5fda-4097-ad10-72b12ae565ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:09.580148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31393efc-5fda-4097-ad10-72b12ae565ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:09.581591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31393efc-5fda-4097-ad10-72b12ae565ea None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:09.583203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 578/1133] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:32:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:09.589255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e4225-b4d0-4c96-93c2-f0843d431bc6 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:09.701022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e4225-b4d0-4c96-93c2-f0843d431bc6 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:09.736501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:09.741945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfda5c6-0127-48d2-979f-3f85e5245c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:09.746365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfda5c6-0127-48d2-979f-3f85e5245c8a None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:09.746365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfda5c6-0127-48d2-979f-3f85e5245c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:09.746365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfda5c6-0127-48d2-979f-3f85e5245c8a None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:09.746365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfda5c6-0127-48d2-979f-3f85e5245c8a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:09.746365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfda5c6-0127-48d2-979f-3f85e5245c8a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:09.861546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 556/1134] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:32:09 2022] POST /identity/v3/projects => generated 311 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:11.830445 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d99590-a490-4119-8a64-e5d760ee05c3 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:11.831050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d99590-a490-4119-8a64-e5d760ee05c3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:11.831050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d99590-a490-4119-8a64-e5d760ee05c3 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:11.832396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 579/1135] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:11.837912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926a7ff-395c-4c3d-b071-688aa1df2c86 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:11.838340 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926a7ff-395c-4c3d-b071-688aa1df2c86 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:11.838474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926a7ff-395c-4c3d-b071-688aa1df2c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:11.857919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f926a7ff-395c-4c3d-b071-688aa1df2c86 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:11.859183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f926a7ff-395c-4c3d-b071-688aa1df2c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:11.929690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 557/1136] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:12.210040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d99590-a490-4119-8a64-e5d760ee05c3 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:12.224337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:12.318913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d99590-a490-4119-8a64-e5d760ee05c3 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:12.354071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fe4de-a980-474f-b653-086691b4ee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:12.354736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fe4de-a980-474f-b653-086691b4ee8b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:12.355037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fe4de-a980-474f-b653-086691b4ee8b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:12.355305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fe4de-a980-474f-b653-086691b4ee8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:12.356440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fe4de-a980-474f-b653-086691b4ee8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:12.357939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fe4de-a980-474f-b653-086691b4ee8b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:12.359537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 580/1137] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:32:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:12.366476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926a7ff-395c-4c3d-b071-688aa1df2c86 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:12.468874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926a7ff-395c-4c3d-b071-688aa1df2c86 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:12.488042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:12.507979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50308b43-8dbf-4776-a958-152df2cb4ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:12.508602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50308b43-8dbf-4776-a958-152df2cb4ef4 None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:12.508883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50308b43-8dbf-4776-a958-152df2cb4ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:12.509160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50308b43-8dbf-4776-a958-152df2cb4ef4 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:12.509821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50308b43-8dbf-4776-a958-152df2cb4ef4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:12.511298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50308b43-8dbf-4776-a958-152df2cb4ef4 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:12.597928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 558/1138] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:32:12 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:14.675514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a92f12-5df9-4908-b7b7-77aa91fd539a None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:14.676073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a92f12-5df9-4908-b7b7-77aa91fd539a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:14.676073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a92f12-5df9-4908-b7b7-77aa91fd539a None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:14.677415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 581/1139] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:14.682846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11aa84-caaa-4d67-818e-ba803c1981f1 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:14.683267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11aa84-caaa-4d67-818e-ba803c1981f1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:14.683431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11aa84-caaa-4d67-818e-ba803c1981f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:14.701987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6f11aa84-caaa-4d67-818e-ba803c1981f1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:14.703233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6f11aa84-caaa-4d67-818e-ba803c1981f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:14.772114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 559/1140] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:15.045863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a92f12-5df9-4908-b7b7-77aa91fd539a None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:15.124197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:15.147006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a92f12-5df9-4908-b7b7-77aa91fd539a None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:15.179778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc02c51-3fea-44aa-83d4-ad6c8e64b0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:15.180405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc02c51-3fea-44aa-83d4-ad6c8e64b0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:15.180687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc02c51-3fea-44aa-83d4-ad6c8e64b0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:15.180961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc02c51-3fea-44aa-83d4-ad6c8e64b0d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:15.191689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc02c51-3fea-44aa-83d4-ad6c8e64b0d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-ecc02c51-3fea-44aa-83d4-ad6c8e64b0d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:32:15.196465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:32:15.197672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:32:15.198999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 582/1141] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:32:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:32:15.203626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11aa84-caaa-4d67-818e-ba803c1981f1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:15.299358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:15.302152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11aa84-caaa-4d67-818e-ba803c1981f1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:15.336960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73486831-7663-4df7-94aa-b6da1b9a567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:15.337571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73486831-7663-4df7-94aa-b6da1b9a567b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:15.337856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73486831-7663-4df7-94aa-b6da1b9a567b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:15.338140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73486831-7663-4df7-94aa-b6da1b9a567b None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:15.338836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73486831-7663-4df7-94aa-b6da1b9a567b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:15.345603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73486831-7663-4df7-94aa-b6da1b9a567b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:15.345603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73486831-7663-4df7-94aa-b6da1b9a567b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:15.349095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 560/1142] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:32:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:15.355047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc02c51-3fea-44aa-83d4-ad6c8e64b0d2 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:15.454409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc02c51-3fea-44aa-83d4-ad6c8e64b0d2 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:15.488157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9868b24-513d-4613-b426-d3bcb7f9d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:15.488788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9868b24-513d-4613-b426-d3bcb7f9d7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:15.489070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9868b24-513d-4613-b426-d3bcb7f9d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:15.489343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9868b24-513d-4613-b426-d3bcb7f9d7f9 None admin] PATH_INFO: `/v3/users/e0e4e2e9113c41589405e855e12f3b12` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:15.497286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:15.503458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9868b24-513d-4613-b426-d3bcb7f9d7f9 None admin] RBAC: Authorizing `identity:get_user(user_id=e0e4e2e9113c41589405e855e12f3b12)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:15.509403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9868b24-513d-4613-b426-d3bcb7f9d7f9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:15.509403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 583/1143] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:15 2022] GET /identity/v3/users/e0e4e2e9113c41589405e855e12f3b12 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:15.512617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73486831-7663-4df7-94aa-b6da1b9a567b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:15.612083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73486831-7663-4df7-94aa-b6da1b9a567b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:15.645295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1e44-5cb0-478a-b192-f5810e951f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:15.645895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1e44-5cb0-478a-b192-f5810e951f17 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:15.646181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1e44-5cb0-478a-b192-f5810e951f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:15.646459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1e44-5cb0-478a-b192-f5810e951f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:15.675661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:15.712592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1e44-5cb0-478a-b192-f5810e951f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:15.714059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1e44-5cb0-478a-b192-f5810e951f17 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:15.811013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 561/1144] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:32:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:32:15.817159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9868b24-513d-4613-b426-d3bcb7f9d7f9 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:15.915733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9868b24-513d-4613-b426-d3bcb7f9d7f9 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:15.949203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540712d-0220-4607-b538-0f949f7e670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:15.949825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540712d-0220-4607-b538-0f949f7e670d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:15.950113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540712d-0220-4607-b538-0f949f7e670d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:15.950386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540712d-0220-4607-b538-0f949f7e670d None admin] PATH_INFO: `/v3/projects/a9eb5965eb404da7b00b9ee895b8a7f6` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:15.951536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540712d-0220-4607-b538-0f949f7e670d None admin] RBAC: Authorizing `identity:get_project(project_id=a9eb5965eb404da7b00b9ee895b8a7f6)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:15.952977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540712d-0220-4607-b538-0f949f7e670d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:15.955275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 584/1145] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:32:15 2022] GET /identity/v3/projects/a9eb5965eb404da7b00b9ee895b8a7f6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:15.961591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1e44-5cb0-478a-b192-f5810e951f17 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:16.022379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:16.068989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1e44-5cb0-478a-b192-f5810e951f17 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:16.102275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727745f-b0a5-4e3a-ae0e-117b06d95450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:16.102899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727745f-b0a5-4e3a-ae0e-117b06d95450 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:16.103184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727745f-b0a5-4e3a-ae0e-117b06d95450 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:16.103452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727745f-b0a5-4e3a-ae0e-117b06d95450 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:16.104169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727745f-b0a5-4e3a-ae0e-117b06d95450 None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=a9eb5965eb404da7b00b9ee895b8a7f6, user.id=e0e4e2e9113c41589405e855e12f3b12) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:16.104471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727745f-b0a5-4e3a-ae0e-117b06d95450 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:16.105884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727745f-b0a5-4e3a-ae0e-117b06d95450 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:16.111164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 562/1146] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:32:15 2022] GET /identity/v3/role_assignments?user.id=e0e4e2e9113c41589405e855e12f3b12&scope.project.id=a9eb5965eb404da7b00b9ee895b8a7f6&role.id=9e904216086e4a06abf9216b0c863f89 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:18.023953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19df6d1-fd86-44a9-af51-205a95c0fa48 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:18.024377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19df6d1-fd86-44a9-af51-205a95c0fa48 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:18.024510 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19df6d1-fd86-44a9-af51-205a95c0fa48 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:18.025880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 585/1147] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:18.031119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65581ed2-1822-43e3-a7d9-fb07e842e02d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:18.031547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65581ed2-1822-43e3-a7d9-fb07e842e02d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:18.031672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65581ed2-1822-43e3-a7d9-fb07e842e02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:18.049942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-65581ed2-1822-43e3-a7d9-fb07e842e02d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:18.051150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-65581ed2-1822-43e3-a7d9-fb07e842e02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:18.117509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 563/1148] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:18.390966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19df6d1-fd86-44a9-af51-205a95c0fa48 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:18.458693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:18.497472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19df6d1-fd86-44a9-af51-205a95c0fa48 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:18.533370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95c24c-5b74-4703-997f-92e535f47a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:18.534001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c24c-5b74-4703-997f-92e535f47a25 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:18.534288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c24c-5b74-4703-997f-92e535f47a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:18.534595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c24c-5b74-4703-997f-92e535f47a25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:18.544422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95c24c-5b74-4703-997f-92e535f47a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-ae95c24c-5b74-4703-997f-92e535f47a25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:32:18.550356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:32:18.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:32:18.552691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 586/1149] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:32:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:32:18.557615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65581ed2-1822-43e3-a7d9-fb07e842e02d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:18.612292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:18.667101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65581ed2-1822-43e3-a7d9-fb07e842e02d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:18.709433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef640e-841a-4145-a0f7-de0a67a8ae28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:18.710041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef640e-841a-4145-a0f7-de0a67a8ae28 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:18.710323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef640e-841a-4145-a0f7-de0a67a8ae28 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:18.710703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef640e-841a-4145-a0f7-de0a67a8ae28 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:18.711369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef640e-841a-4145-a0f7-de0a67a8ae28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:18.711675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef640e-841a-4145-a0f7-de0a67a8ae28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:18.713053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef640e-841a-4145-a0f7-de0a67a8ae28 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:18.721161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 564/1150] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:32:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:18.727119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95c24c-5b74-4703-997f-92e535f47a25 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:18.826453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95c24c-5b74-4703-997f-92e535f47a25 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:18.850121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:18.861575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942051f-cc69-4a62-a166-f51da32b5ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:18.862688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942051f-cc69-4a62-a166-f51da32b5ddb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:18.862688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942051f-cc69-4a62-a166-f51da32b5ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:18.862870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942051f-cc69-4a62-a166-f51da32b5ddb None admin] PATH_INFO: `/v3/users/e0e4e2e9113c41589405e855e12f3b12` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:18.863869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942051f-cc69-4a62-a166-f51da32b5ddb None admin] RBAC: Authorizing `identity:get_user(user_id=e0e4e2e9113c41589405e855e12f3b12)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:18.865310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942051f-cc69-4a62-a166-f51da32b5ddb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:18.867178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 587/1151] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:18 2022] GET /identity/v3/users/e0e4e2e9113c41589405e855e12f3b12 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:18.873323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef640e-841a-4145-a0f7-de0a67a8ae28 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:18.976042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef640e-841a-4145-a0f7-de0a67a8ae28 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:18.992950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:19.010378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68297502-dea2-4395-826a-70845550af7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:19.011063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68297502-dea2-4395-826a-70845550af7a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:19.011351 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68297502-dea2-4395-826a-70845550af7a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:19.011623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68297502-dea2-4395-826a-70845550af7a None admin] PATH_INFO: `/v3/projects/a9eb5965eb404da7b00b9ee895b8a7f6` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:19.012846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68297502-dea2-4395-826a-70845550af7a None admin] RBAC: Authorizing `identity:get_project(project_id=a9eb5965eb404da7b00b9ee895b8a7f6)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:19.014298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68297502-dea2-4395-826a-70845550af7a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:19.016204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 565/1152] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:32:18 2022] GET /identity/v3/projects/a9eb5965eb404da7b00b9ee895b8a7f6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:19.022307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942051f-cc69-4a62-a166-f51da32b5ddb None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:19.124380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942051f-cc69-4a62-a166-f51da32b5ddb None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:19.160351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76eb1-a028-4769-b76b-f976b09a6af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:19.161010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76eb1-a028-4769-b76b-f976b09a6af6 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:19.161289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76eb1-a028-4769-b76b-f976b09a6af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:19.161563 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76eb1-a028-4769-b76b-f976b09a6af6 None admin] PATH_INFO: `/v3/projects/a9eb5965eb404da7b00b9ee895b8a7f6/users/e0e4e2e9113c41589405e855e12f3b12/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:19.163501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76eb1-a028-4769-b76b-f976b09a6af6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9eb5965eb404da7b00b9ee895b8a7f6, user_id=e0e4e2e9113c41589405e855e12f3b12, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:19.164958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76eb1-a028-4769-b76b-f976b09a6af6 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:19.283839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 588/1153] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:32:19 2022] PUT /identity/v3/projects/a9eb5965eb404da7b00b9ee895b8a7f6/users/e0e4e2e9113c41589405e855e12f3b12/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:21.186303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153fe1b-2f17-49bf-88ac-c5546c163b21 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:21.187064 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153fe1b-2f17-49bf-88ac-c5546c163b21 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:21.187064 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153fe1b-2f17-49bf-88ac-c5546c163b21 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:21.189094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 566/1154] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:21.194619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02876-fcc1-44ed-b198-2feeacc796ab None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:21.195057 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02876-fcc1-44ed-b198-2feeacc796ab None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:21.195184 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02876-fcc1-44ed-b198-2feeacc796ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:21.214716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-aff02876-fcc1-44ed-b198-2feeacc796ab None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:21.215917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-aff02876-fcc1-44ed-b198-2feeacc796ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:21.264366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:21.286674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 589/1155] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:21.554683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153fe1b-2f17-49bf-88ac-c5546c163b21 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:21.654471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153fe1b-2f17-49bf-88ac-c5546c163b21 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:21.687638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3a292-bc00-499b-8e8a-c6d2a6d9a84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:21.688848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3a292-bc00-499b-8e8a-c6d2a6d9a84b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:21.689140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3a292-bc00-499b-8e8a-c6d2a6d9a84b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:21.689414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3a292-bc00-499b-8e8a-c6d2a6d9a84b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:21.699315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3a292-bc00-499b-8e8a-c6d2a6d9a84b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-b4e3a292-bc00-499b-8e8a-c6d2a6d9a84b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:32:21.704196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:32:21.705329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:32:21.706266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 567/1156] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:32:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:32:21.711675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff02876-fcc1-44ed-b198-2feeacc796ab None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:21.808393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff02876-fcc1-44ed-b198-2feeacc796ab None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:21.841271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be84f-9930-4dcd-b138-e5edb3a87422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:21.841872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be84f-9930-4dcd-b138-e5edb3a87422 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:21.842153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be84f-9930-4dcd-b138-e5edb3a87422 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:21.842424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be84f-9930-4dcd-b138-e5edb3a87422 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:21.843111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92be84f-9930-4dcd-b138-e5edb3a87422 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:21.843416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92be84f-9930-4dcd-b138-e5edb3a87422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:21.845535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92be84f-9930-4dcd-b138-e5edb3a87422 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:21.854669 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 590/1157] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:32:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:21.860573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3a292-bc00-499b-8e8a-c6d2a6d9a84b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:21.907826 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:21.960623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3a292-bc00-499b-8e8a-c6d2a6d9a84b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:21.995700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8825da-9664-46e4-b473-c10ff7ba12ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:21.996337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8825da-9664-46e4-b473-c10ff7ba12ce None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:21.996623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8825da-9664-46e4-b473-c10ff7ba12ce None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:21.996899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8825da-9664-46e4-b473-c10ff7ba12ce None admin] PATH_INFO: `/v3/users/e0e4e2e9113c41589405e855e12f3b12` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:21.998040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8825da-9664-46e4-b473-c10ff7ba12ce None admin] RBAC: Authorizing `identity:get_user(user_id=e0e4e2e9113c41589405e855e12f3b12)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:21.999521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8825da-9664-46e4-b473-c10ff7ba12ce None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:22.003032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 568/1158] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:21 2022] GET /identity/v3/users/e0e4e2e9113c41589405e855e12f3b12 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:22.012661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be84f-9930-4dcd-b138-e5edb3a87422 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:22.068736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:22.122059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be84f-9930-4dcd-b138-e5edb3a87422 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:22.159500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e5964-618e-4999-9bc4-38bda9280987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:22.160145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e5964-618e-4999-9bc4-38bda9280987 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:22.160432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e5964-618e-4999-9bc4-38bda9280987 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:22.160707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e5964-618e-4999-9bc4-38bda9280987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:22.230807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e5964-618e-4999-9bc4-38bda9280987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:22.232272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e5964-618e-4999-9bc4-38bda9280987 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:22.278465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:22.334773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 591/1159] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:32:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:32:22.342786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8825da-9664-46e4-b473-c10ff7ba12ce None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:22.450633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8825da-9664-46e4-b473-c10ff7ba12ce None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:22.464527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:22.483922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f0fc-a5d6-46e0-8828-5016ae3aa407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:22.484545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f0fc-a5d6-46e0-8828-5016ae3aa407 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:22.484828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f0fc-a5d6-46e0-8828-5016ae3aa407 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:22.485696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f0fc-a5d6-46e0-8828-5016ae3aa407 None admin] PATH_INFO: `/v3/projects/a9eb5965eb404da7b00b9ee895b8a7f6` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:22.487039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1f0fc-a5d6-46e0-8828-5016ae3aa407 None admin] RBAC: Authorizing `identity:get_project(project_id=a9eb5965eb404da7b00b9ee895b8a7f6)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:22.488483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1f0fc-a5d6-46e0-8828-5016ae3aa407 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:22.490108 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 569/1160] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:32:22 2022] GET /identity/v3/projects/a9eb5965eb404da7b00b9ee895b8a7f6 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:22.496766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e5964-618e-4999-9bc4-38bda9280987 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:22.594075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e5964-618e-4999-9bc4-38bda9280987 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:22.628397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de758995-47f2-4ef4-a5b2-0ae1977725fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:22.629007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de758995-47f2-4ef4-a5b2-0ae1977725fe None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:22.629288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de758995-47f2-4ef4-a5b2-0ae1977725fe None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:22.629571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de758995-47f2-4ef4-a5b2-0ae1977725fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:22.630299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de758995-47f2-4ef4-a5b2-0ae1977725fe None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=a9eb5965eb404da7b00b9ee895b8a7f6, user.id=e0e4e2e9113c41589405e855e12f3b12) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:22.630646 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de758995-47f2-4ef4-a5b2-0ae1977725fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:22.632114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de758995-47f2-4ef4-a5b2-0ae1977725fe None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:22.637854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 592/1161] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:32:22 2022] GET /identity/v3/role_assignments?user.id=e0e4e2e9113c41589405e855e12f3b12&scope.project.id=a9eb5965eb404da7b00b9ee895b8a7f6&role.id=9e904216086e4a06abf9216b0c863f89 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:24.577459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345625e-b515-41f8-8289-a794dad261f1 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:24.578031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345625e-b515-41f8-8289-a794dad261f1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:24.578031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345625e-b515-41f8-8289-a794dad261f1 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:24.580160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 570/1162] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:24.585697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dbbdd-5408-48d4-bb1b-40f618fc6f5c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:24.586156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dbbdd-5408-48d4-bb1b-40f618fc6f5c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:24.586248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dbbdd-5408-48d4-bb1b-40f618fc6f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:24.604661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9f9dbbdd-5408-48d4-bb1b-40f618fc6f5c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:24.605843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9f9dbbdd-5408-48d4-bb1b-40f618fc6f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:24.607828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:24.675718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 593/1163] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:24.976814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345625e-b515-41f8-8289-a794dad261f1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:25.093470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345625e-b515-41f8-8289-a794dad261f1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:25.130616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1db7f-95df-4e85-93da-4a5feb29aef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:25.131230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1db7f-95df-4e85-93da-4a5feb29aef6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:25.131514 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1db7f-95df-4e85-93da-4a5feb29aef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:25.131787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1db7f-95df-4e85-93da-4a5feb29aef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:25.196293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:25.224371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1db7f-95df-4e85-93da-4a5feb29aef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:25.227831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1db7f-95df-4e85-93da-4a5feb29aef6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:25.355235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 571/1164] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:32:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:32:25.362185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dbbdd-5408-48d4-bb1b-40f618fc6f5c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:25.483959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dbbdd-5408-48d4-bb1b-40f618fc6f5c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:25.528895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b989f-81f2-4a87-b93e-dca402fdef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:25.529533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b989f-81f2-4a87-b93e-dca402fdef67 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:25.529817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b989f-81f2-4a87-b93e-dca402fdef67 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:25.530086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b989f-81f2-4a87-b93e-dca402fdef67 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:25.543010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b989f-81f2-4a87-b93e-dca402fdef67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:25.545857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b989f-81f2-4a87-b93e-dca402fdef67 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:25.549414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-ad0b989f-81f2-4a87-b93e-dca402fdef67 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:32:25.564227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:32:25.570387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:32:25.571615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 594/1165] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:32:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:32:25.572562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1db7f-95df-4e85-93da-4a5feb29aef6 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:25.668428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:25.686297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1db7f-95df-4e85-93da-4a5feb29aef6 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:25.722022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012e270-0859-4060-a85c-cac848c46f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:25.722658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012e270-0859-4060-a85c-cac848c46f5d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:25.722940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012e270-0859-4060-a85c-cac848c46f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:25.723213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012e270-0859-4060-a85c-cac848c46f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:25.723861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012e270-0859-4060-a85c-cac848c46f5d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:25.724165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012e270-0859-4060-a85c-cac848c46f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:25.725545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012e270-0859-4060-a85c-cac848c46f5d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:25.733390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 572/1166] 10.222.0.82 () {56 vars in 1115 bytes} [Wed Aug 24 03:32:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:32:27.745750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0e093-076d-4b57-8832-398875f4beaa None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:27.746344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0e093-076d-4b57-8832-398875f4beaa None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:27.746344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0e093-076d-4b57-8832-398875f4beaa None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:27.747678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 595/1167] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:32:27.753572 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8e40b-5b7d-4c42-bb8a-7f93950e3659 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:27.753989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8e40b-5b7d-4c42-bb8a-7f93950e3659 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:27.754123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8e40b-5b7d-4c42-bb8a-7f93950e3659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:27.772254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-faa8e40b-5b7d-4c42-bb8a-7f93950e3659 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:27.773433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-faa8e40b-5b7d-4c42-bb8a-7f93950e3659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:27.851378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 573/1168] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:28.128303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0e093-076d-4b57-8832-398875f4beaa None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:28.250811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0e093-076d-4b57-8832-398875f4beaa None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:28.289163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56690c7a-861b-49aa-8983-7a852ed5b49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:28.289701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56690c7a-861b-49aa-8983-7a852ed5b49d None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:28.289982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56690c7a-861b-49aa-8983-7a852ed5b49d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:28.290254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56690c7a-861b-49aa-8983-7a852ed5b49d None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:28.290962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56690c7a-861b-49aa-8983-7a852ed5b49d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:28.292336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56690c7a-861b-49aa-8983-7a852ed5b49d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:28.376117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 596/1169] 10.222.0.82 () {60 vars in 1138 bytes} [Wed Aug 24 03:32:28 2022] POST /identity/v3/domains => generated 267 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:30.372048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4432328-8f8d-4b7f-bebe-75bc2b0c0990 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:30.372565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4432328-8f8d-4b7f-bebe-75bc2b0c0990 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:30.372565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4432328-8f8d-4b7f-bebe-75bc2b0c0990 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:30.373995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 574/1170] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:30.380694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c5658-bae8-4c17-bf14-209e3470faeb None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:30.381170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c5658-bae8-4c17-bf14-209e3470faeb None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:30.381259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c5658-bae8-4c17-bf14-209e3470faeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:30.400854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9d8c5658-bae8-4c17-bf14-209e3470faeb None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:30.402076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9d8c5658-bae8-4c17-bf14-209e3470faeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:30.462465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:30.476431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 597/1171] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:32:30.754521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4432328-8f8d-4b7f-bebe-75bc2b0c0990 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:30.843194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:30.870103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4432328-8f8d-4b7f-bebe-75bc2b0c0990 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:30.906131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a2c69-fe28-45d3-b2d9-c662b35ec6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:30.955268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a2c69-fe28-45d3-b2d9-c662b35ec6cc None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:30.955268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a2c69-fe28-45d3-b2d9-c662b35ec6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:30.955268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a2c69-fe28-45d3-b2d9-c662b35ec6cc None admin] PATH_INFO: `/v3/domains/58a2f989d01c4b47b4a11c1714593a6b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:30.955268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a2c69-fe28-45d3-b2d9-c662b35ec6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=58a2f989d01c4b47b4a11c1714593a6b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:30.955268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a2c69-fe28-45d3-b2d9-c662b35ec6cc None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:30.955268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 575/1172] 10.222.0.82 () {56 vars in 1183 bytes} [Wed Aug 24 03:32:30 2022] GET /identity/v3/domains/58a2f989d01c4b47b4a11c1714593a6b => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:30.956057 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c5658-bae8-4c17-bf14-209e3470faeb None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:31.036726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c5658-bae8-4c17-bf14-209e3470faeb None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:31.074747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5333-04fd-426f-bf01-6f99dde58d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:31.075357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5333-04fd-426f-bf01-6f99dde58d86 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:31.075635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5333-04fd-426f-bf01-6f99dde58d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:31.075905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5333-04fd-426f-bf01-6f99dde58d86 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:31.076543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee5333-04fd-426f-bf01-6f99dde58d86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:31.077986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee5333-04fd-426f-bf01-6f99dde58d86 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:31.169977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 598/1173] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:32:30 2022] POST /identity/v3/projects => generated 361 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:33.159863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fa526-5f1a-4c3b-b493-368776f386d6 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.160429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fa526-5f1a-4c3b-b493-368776f386d6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.160429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fa526-5f1a-4c3b-b493-368776f386d6 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:33.161758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 576/1174] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:33.167281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5385fcf-6711-40de-9b68-363e5131f04c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.167710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5385fcf-6711-40de-9b68-363e5131f04c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.167839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5385fcf-6711-40de-9b68-363e5131f04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:33.186330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d5385fcf-6711-40de-9b68-363e5131f04c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:33.187566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d5385fcf-6711-40de-9b68-363e5131f04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:33.221043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:33.254514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 599/1175] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:33.528694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fa526-5f1a-4c3b-b493-368776f386d6 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:33.642721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fa526-5f1a-4c3b-b493-368776f386d6 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:33.677840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157a8d7-02ca-4936-b997-394193d190d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:33.678479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157a8d7-02ca-4936-b997-394193d190d6 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.678791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157a8d7-02ca-4936-b997-394193d190d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.679071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157a8d7-02ca-4936-b997-394193d190d6 None admin] PATH_INFO: `/v3/domains/58a2f989d01c4b47b4a11c1714593a6b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:33.680212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157a8d7-02ca-4936-b997-394193d190d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=58a2f989d01c4b47b4a11c1714593a6b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:33.681662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157a8d7-02ca-4936-b997-394193d190d6 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:33.683287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 577/1176] 10.222.0.82 () {56 vars in 1183 bytes} [Wed Aug 24 03:32:33 2022] GET /identity/v3/domains/58a2f989d01c4b47b4a11c1714593a6b => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:33.689757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5385fcf-6711-40de-9b68-363e5131f04c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:33.800261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5385fcf-6711-40de-9b68-363e5131f04c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:33.833999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c5009-c136-4e4e-a2cd-383a745ec99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:33.834637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c5009-c136-4e4e-a2cd-383a745ec99e None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.834926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c5009-c136-4e4e-a2cd-383a745ec99e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.835199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c5009-c136-4e4e-a2cd-383a745ec99e None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:33.835852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c5009-c136-4e4e-a2cd-383a745ec99e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:33.837290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c5009-c136-4e4e-a2cd-383a745ec99e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:33.947497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 600/1177] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:32:33 2022] POST /identity/v3/users => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:32:35.974421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38ffb4-fc37-4e9d-a2fe-e060cff935d5 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:35.975023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38ffb4-fc37-4e9d-a2fe-e060cff935d5 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:35.975123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38ffb4-fc37-4e9d-a2fe-e060cff935d5 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:35.976447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 578/1178] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:35.982029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7c976-7bb3-42b5-96bc-cc07928ac495 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:35.982491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7c976-7bb3-42b5-96bc-cc07928ac495 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:35.982629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7c976-7bb3-42b5-96bc-cc07928ac495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:36.002141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-74b7c976-7bb3-42b5-96bc-cc07928ac495 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:36.003404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-74b7c976-7bb3-42b5-96bc-cc07928ac495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:36.024897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:36.073396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 601/1179] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:36.351985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38ffb4-fc37-4e9d-a2fe-e060cff935d5 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:36.355892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:36.455352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38ffb4-fc37-4e9d-a2fe-e060cff935d5 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:36.489974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc43cd-9067-40b8-86f5-ac2b1882a2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:36.490639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc43cd-9067-40b8-86f5-ac2b1882a2bf None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:36.491015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc43cd-9067-40b8-86f5-ac2b1882a2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:36.491303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc43cd-9067-40b8-86f5-ac2b1882a2bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:36.501830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc43cd-9067-40b8-86f5-ac2b1882a2bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-83cc43cd-9067-40b8-86f5-ac2b1882a2bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:32:36.506712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:32:36.507711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:32:36.508850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 579/1180] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:32:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:32:36.514035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7c976-7bb3-42b5-96bc-cc07928ac495 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:36.624633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7c976-7bb3-42b5-96bc-cc07928ac495 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:36.659257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bd55a-c495-4e1c-951c-5a2709027abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:36.659867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bd55a-c495-4e1c-951c-5a2709027abd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:36.660148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bd55a-c495-4e1c-951c-5a2709027abd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:36.660424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bd55a-c495-4e1c-951c-5a2709027abd None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:36.661082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bd55a-c495-4e1c-951c-5a2709027abd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:36.661383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bd55a-c495-4e1c-951c-5a2709027abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:36.662804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bd55a-c495-4e1c-951c-5a2709027abd None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:36.671254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 602/1181] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:32:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:36.677265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc43cd-9067-40b8-86f5-ac2b1882a2bf None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:36.740393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:36.778858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc43cd-9067-40b8-86f5-ac2b1882a2bf None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:36.819939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76915077-17c5-402b-8b9f-c7f7847cdcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:36.820866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76915077-17c5-402b-8b9f-c7f7847cdcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:36.821251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76915077-17c5-402b-8b9f-c7f7847cdcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:36.821668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76915077-17c5-402b-8b9f-c7f7847cdcf8 None admin] PATH_INFO: `/v3/users/e36ebb44b40d416684aa067bd3fd519e` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:36.835756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76915077-17c5-402b-8b9f-c7f7847cdcf8 None admin] RBAC: Authorizing `identity:get_user(user_id=e36ebb44b40d416684aa067bd3fd519e)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:36.837226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76915077-17c5-402b-8b9f-c7f7847cdcf8 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:36.838907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 580/1182] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:36 2022] GET /identity/v3/users/e36ebb44b40d416684aa067bd3fd519e => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:36.845776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bd55a-c495-4e1c-951c-5a2709027abd None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:36.865141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:36.947035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bd55a-c495-4e1c-951c-5a2709027abd None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:36.981344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b25be-7d34-44c7-950c-25cb073ff823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:36.981962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b25be-7d34-44c7-950c-25cb073ff823 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:36.982254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b25be-7d34-44c7-950c-25cb073ff823 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:36.982824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b25be-7d34-44c7-950c-25cb073ff823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:37.051747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b25be-7d34-44c7-950c-25cb073ff823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:37.053222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b25be-7d34-44c7-950c-25cb073ff823 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:37.073166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:37.153869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 603/1183] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:32:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:32:37.160146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76915077-17c5-402b-8b9f-c7f7847cdcf8 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:37.262323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76915077-17c5-402b-8b9f-c7f7847cdcf8 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:37.285587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:37.297213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae7b1f-5ec3-40b4-99e2-11e70604e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:37.298358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae7b1f-5ec3-40b4-99e2-11e70604e3ba None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:37.298661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae7b1f-5ec3-40b4-99e2-11e70604e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:37.298934 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae7b1f-5ec3-40b4-99e2-11e70604e3ba None admin] PATH_INFO: `/v3/projects/8d2688645582412cba09fc83e7396570` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:37.300098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae7b1f-5ec3-40b4-99e2-11e70604e3ba None admin] RBAC: Authorizing `identity:get_project(project_id=8d2688645582412cba09fc83e7396570)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:37.301554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae7b1f-5ec3-40b4-99e2-11e70604e3ba None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:37.303208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 581/1184] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:32:37 2022] GET /identity/v3/projects/8d2688645582412cba09fc83e7396570 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:37.309675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b25be-7d34-44c7-950c-25cb073ff823 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:37.409367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b25be-7d34-44c7-950c-25cb073ff823 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:37.426785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:37.444374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea37295-78e5-4ef0-80a7-b2b07e077e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:37.444985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea37295-78e5-4ef0-80a7-b2b07e077e22 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:37.445266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea37295-78e5-4ef0-80a7-b2b07e077e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:37.445572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea37295-78e5-4ef0-80a7-b2b07e077e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:37.447206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea37295-78e5-4ef0-80a7-b2b07e077e22 None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=8d2688645582412cba09fc83e7396570, user.id=e36ebb44b40d416684aa067bd3fd519e) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:37.447589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea37295-78e5-4ef0-80a7-b2b07e077e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:37.448935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea37295-78e5-4ef0-80a7-b2b07e077e22 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:37.456298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 604/1185] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:32:37 2022] GET /identity/v3/role_assignments?user.id=e36ebb44b40d416684aa067bd3fd519e&scope.project.id=8d2688645582412cba09fc83e7396570&role.id=9e904216086e4a06abf9216b0c863f89 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:39.369089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335d8fb-8478-46a5-8fa3-49bcdc12cd72 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:39.369644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335d8fb-8478-46a5-8fa3-49bcdc12cd72 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:39.369644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335d8fb-8478-46a5-8fa3-49bcdc12cd72 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:39.370912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 582/1186] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:39.376208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f56061-ce31-4f21-9d00-27b475fc36d0 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:39.376738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f56061-ce31-4f21-9d00-27b475fc36d0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:39.376738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f56061-ce31-4f21-9d00-27b475fc36d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:39.395328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b7f56061-ce31-4f21-9d00-27b475fc36d0 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:39.396522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b7f56061-ce31-4f21-9d00-27b475fc36d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:39.463574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 605/1187] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:39.731164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335d8fb-8478-46a5-8fa3-49bcdc12cd72 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:39.830241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335d8fb-8478-46a5-8fa3-49bcdc12cd72 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:39.864234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f3b40-76cd-41fc-b076-c50f78ef7321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:39.864849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f3b40-76cd-41fc-b076-c50f78ef7321 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:39.865129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f3b40-76cd-41fc-b076-c50f78ef7321 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:39.865402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f3b40-76cd-41fc-b076-c50f78ef7321 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:39.875399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f3b40-76cd-41fc-b076-c50f78ef7321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-a24f3b40-76cd-41fc-b076-c50f78ef7321 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:32:39.880603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:32:39.881822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:32:39.882836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 583/1188] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:32:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:32:39.887732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f56061-ce31-4f21-9d00-27b475fc36d0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:39.986665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f56061-ce31-4f21-9d00-27b475fc36d0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.000086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:40.021334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93cee0-e36e-4431-99df-6ac4b65930d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.021950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93cee0-e36e-4431-99df-6ac4b65930d2 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.022230 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93cee0-e36e-4431-99df-6ac4b65930d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.022523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93cee0-e36e-4431-99df-6ac4b65930d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.023200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93cee0-e36e-4431-99df-6ac4b65930d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:40.023505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93cee0-e36e-4431-99df-6ac4b65930d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.024940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93cee0-e36e-4431-99df-6ac4b65930d2 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.032981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 606/1189] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:32:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:40.038860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f3b40-76cd-41fc-b076-c50f78ef7321 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:40.090283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:40.139756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f3b40-76cd-41fc-b076-c50f78ef7321 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.173923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225705e-894e-4bcc-8326-134706554be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.174567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225705e-894e-4bcc-8326-134706554be9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.174855 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225705e-894e-4bcc-8326-134706554be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.175135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225705e-894e-4bcc-8326-134706554be9 None admin] PATH_INFO: `/v3/users/e36ebb44b40d416684aa067bd3fd519e` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.176262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225705e-894e-4bcc-8326-134706554be9 None admin] RBAC: Authorizing `identity:get_user(user_id=e36ebb44b40d416684aa067bd3fd519e)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.177710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225705e-894e-4bcc-8326-134706554be9 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.179385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 584/1190] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:40 2022] GET /identity/v3/users/e36ebb44b40d416684aa067bd3fd519e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:40.185247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93cee0-e36e-4431-99df-6ac4b65930d2 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:40.283685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93cee0-e36e-4431-99df-6ac4b65930d2 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.317258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b106a31-a97d-4581-a9af-382736f2209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.317886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b106a31-a97d-4581-a9af-382736f2209f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.318165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b106a31-a97d-4581-a9af-382736f2209f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.318438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b106a31-a97d-4581-a9af-382736f2209f None admin] PATH_INFO: `/v3/projects/8d2688645582412cba09fc83e7396570` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.319582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b106a31-a97d-4581-a9af-382736f2209f None admin] RBAC: Authorizing `identity:get_project(project_id=8d2688645582412cba09fc83e7396570)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.321022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b106a31-a97d-4581-a9af-382736f2209f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.322684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 607/1191] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:32:40 2022] GET /identity/v3/projects/8d2688645582412cba09fc83e7396570 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:40.328699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225705e-894e-4bcc-8326-134706554be9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:40.430688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225705e-894e-4bcc-8326-134706554be9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:40.448167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:40.464554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24842f-be6a-4aed-b918-88c19a75004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:40.465739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24842f-be6a-4aed-b918-88c19a75004c None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:40.465739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24842f-be6a-4aed-b918-88c19a75004c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:40.465739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24842f-be6a-4aed-b918-88c19a75004c None admin] PATH_INFO: `/v3/projects/8d2688645582412cba09fc83e7396570/users/e36ebb44b40d416684aa067bd3fd519e/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:40.467939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24842f-be6a-4aed-b918-88c19a75004c None admin] RBAC: Authorizing `identity:create_grant(project_id=8d2688645582412cba09fc83e7396570, user_id=e36ebb44b40d416684aa067bd3fd519e, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:40.469399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24842f-be6a-4aed-b918-88c19a75004c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:40.563958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 585/1192] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:32:40 2022] PUT /identity/v3/projects/8d2688645582412cba09fc83e7396570/users/e36ebb44b40d416684aa067bd3fd519e/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:42.482149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af188e8-4623-49c0-9716-792ebd4a34a4 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.482750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af188e8-4623-49c0-9716-792ebd4a34a4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.482750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af188e8-4623-49c0-9716-792ebd4a34a4 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.484080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 608/1193] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:42.490243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463152b6-7eff-495f-9eb4-1bcf8fbc25b5 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:42.490760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463152b6-7eff-495f-9eb4-1bcf8fbc25b5 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:42.490911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463152b6-7eff-495f-9eb4-1bcf8fbc25b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:42.509927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-463152b6-7eff-495f-9eb4-1bcf8fbc25b5 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:42.511167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-463152b6-7eff-495f-9eb4-1bcf8fbc25b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:42.583349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 586/1194] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:32:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:32:42.927242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af188e8-4623-49c0-9716-792ebd4a34a4 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:42.986621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:43.054712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af188e8-4623-49c0-9716-792ebd4a34a4 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.096029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61684927-7cf2-4ecb-878f-741f50f75f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.096682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61684927-7cf2-4ecb-878f-741f50f75f54 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.096965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61684927-7cf2-4ecb-878f-741f50f75f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.097240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61684927-7cf2-4ecb-878f-741f50f75f54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.110421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61684927-7cf2-4ecb-878f-741f50f75f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-61684927-7cf2-4ecb-878f-741f50f75f54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:32:43.116299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:32:43.118081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:32:43.119921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 609/1195] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:32:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:32:43.125473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463152b6-7eff-495f-9eb4-1bcf8fbc25b5 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:43.254431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463152b6-7eff-495f-9eb4-1bcf8fbc25b5 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.268460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:43.298574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16945354-8b74-4348-a0c1-537dceb13187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.299276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16945354-8b74-4348-a0c1-537dceb13187 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.299729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16945354-8b74-4348-a0c1-537dceb13187 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.300038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16945354-8b74-4348-a0c1-537dceb13187 None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.301192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16945354-8b74-4348-a0c1-537dceb13187 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:43.301513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16945354-8b74-4348-a0c1-537dceb13187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:43.302936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16945354-8b74-4348-a0c1-537dceb13187 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:43.312768 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 587/1196] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:32:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:43.319892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61684927-7cf2-4ecb-878f-741f50f75f54 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:43.437222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61684927-7cf2-4ecb-878f-741f50f75f54 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.438991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:43.476152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2c741-4398-41df-8ba1-09e903b6f17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.476814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2c741-4398-41df-8ba1-09e903b6f17b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.477089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2c741-4398-41df-8ba1-09e903b6f17b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.477360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2c741-4398-41df-8ba1-09e903b6f17b None admin] PATH_INFO: `/v3/users/e36ebb44b40d416684aa067bd3fd519e` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.478766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2c741-4398-41df-8ba1-09e903b6f17b None admin] RBAC: Authorizing `identity:get_user(user_id=e36ebb44b40d416684aa067bd3fd519e)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:43.480271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2c741-4398-41df-8ba1-09e903b6f17b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:43.483273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 610/1197] 10.222.0.82 () {56 vars in 1177 bytes} [Wed Aug 24 03:32:43 2022] GET /identity/v3/users/e36ebb44b40d416684aa067bd3fd519e => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:32:43.489712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16945354-8b74-4348-a0c1-537dceb13187 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:43.610369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16945354-8b74-4348-a0c1-537dceb13187 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.647526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144816-abd7-4837-b574-0a9034e144dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.648504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144816-abd7-4837-b574-0a9034e144dd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.649005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144816-abd7-4837-b574-0a9034e144dd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.649513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144816-abd7-4837-b574-0a9034e144dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.693249 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:43.721884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44144816-abd7-4837-b574-0a9034e144dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:43.723376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44144816-abd7-4837-b574-0a9034e144dd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:43.841688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 588/1198] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:32:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:32:43.848397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2c741-4398-41df-8ba1-09e903b6f17b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:43.951689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2c741-4398-41df-8ba1-09e903b6f17b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:43.988700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2f5ec-f3f2-40c2-98de-b83cf66bc38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:43.989330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2f5ec-f3f2-40c2-98de-b83cf66bc38a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:43.989611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2f5ec-f3f2-40c2-98de-b83cf66bc38a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:43.989888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2f5ec-f3f2-40c2-98de-b83cf66bc38a None admin] PATH_INFO: `/v3/projects/8d2688645582412cba09fc83e7396570` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:43.991236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2f5ec-f3f2-40c2-98de-b83cf66bc38a None admin] RBAC: Authorizing `identity:get_project(project_id=8d2688645582412cba09fc83e7396570)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:43.992927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2f5ec-f3f2-40c2-98de-b83cf66bc38a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:43.994631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 611/1199] 10.222.0.82 () {56 vars in 1186 bytes} [Wed Aug 24 03:32:43 2022] GET /identity/v3/projects/8d2688645582412cba09fc83e7396570 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:44.001339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144816-abd7-4837-b574-0a9034e144dd None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:44.072209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:44.103722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144816-abd7-4837-b574-0a9034e144dd None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:44.137357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6adb9a-8f57-4d23-8641-42c7ef2d4dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:44.137953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6adb9a-8f57-4d23-8641-42c7ef2d4dcd None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:44.138231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6adb9a-8f57-4d23-8641-42c7ef2d4dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:44.138523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6adb9a-8f57-4d23-8641-42c7ef2d4dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:44.139254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6adb9a-8f57-4d23-8641-42c7ef2d4dcd None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=8d2688645582412cba09fc83e7396570, user.id=e36ebb44b40d416684aa067bd3fd519e) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:44.139556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6adb9a-8f57-4d23-8641-42c7ef2d4dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:44.141006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6adb9a-8f57-4d23-8641-42c7ef2d4dcd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:44.147656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 589/1200] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:32:43 2022] GET /identity/v3/role_assignments?user.id=e36ebb44b40d416684aa067bd3fd519e&scope.project.id=8d2688645582412cba09fc83e7396570&role.id=9e904216086e4a06abf9216b0c863f89 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:32:53.983558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=81 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:54.144943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=33 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:03.921936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be4eaa-9e7c-4925-83f5-9fabe735ccfa None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:03.922588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be4eaa-9e7c-4925-83f5-9fabe735ccfa None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:03.922588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be4eaa-9e7c-4925-83f5-9fabe735ccfa None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:03.924544 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 612/1201] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:03.930031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef5b3c-8bb8-4f5b-875a-9ae90832c1c1 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:03.934911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef5b3c-8bb8-4f5b-875a-9ae90832c1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:03.934911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef5b3c-8bb8-4f5b-875a-9ae90832c1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:03.950120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-40ef5b3c-8bb8-4f5b-875a-9ae90832c1c1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:03.951360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-40ef5b3c-8bb8-4f5b-875a-9ae90832c1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:04.033755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 590/1202] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:04.315582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be4eaa-9e7c-4925-83f5-9fabe735ccfa None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:04.426836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be4eaa-9e7c-4925-83f5-9fabe735ccfa None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:04.460416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8b1a1-a049-43c7-9782-adc31f78f36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:04.461048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8b1a1-a049-43c7-9782-adc31f78f36d None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:04.461330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8b1a1-a049-43c7-9782-adc31f78f36d None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:04.461603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8b1a1-a049-43c7-9782-adc31f78f36d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:04.472167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8b1a1-a049-43c7-9782-adc31f78f36d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:04.473633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8b1a1-a049-43c7-9782-adc31f78f36d None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-a5f8b1a1-a049-43c7-9782-adc31f78f36d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:04.488585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:04.489794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:04.491031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:04.491031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:04.491031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:04.494677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 613/1203] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:04.495751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef5b3c-8bb8-4f5b-875a-9ae90832c1c1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:04.593166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef5b3c-8bb8-4f5b-875a-9ae90832c1c1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:04.622551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:04.628529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002c0f1-e088-42b2-9eb6-98d9bade4b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:04.629167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c0f1-e088-42b2-9eb6-98d9bade4b75 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:04.629452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c0f1-e088-42b2-9eb6-98d9bade4b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:04.629724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002c0f1-e088-42b2-9eb6-98d9bade4b75 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:04.630383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002c0f1-e088-42b2-9eb6-98d9bade4b75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:04.630705 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002c0f1-e088-42b2-9eb6-98d9bade4b75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:04.632083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002c0f1-e088-42b2-9eb6-98d9bade4b75 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:04.643941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 591/1204] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:34:04.649866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8b1a1-a049-43c7-9782-adc31f78f36d None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:04.702164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:04.749678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8b1a1-a049-43c7-9782-adc31f78f36d None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:04.784912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5abcb3-f636-45a6-928a-ef444ee7c1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:04.785512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5abcb3-f636-45a6-928a-ef444ee7c1b1 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:04.785789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5abcb3-f636-45a6-928a-ef444ee7c1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:04.786061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5abcb3-f636-45a6-928a-ef444ee7c1b1 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:04.786731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5abcb3-f636-45a6-928a-ef444ee7c1b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:04.788214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5abcb3-f636-45a6-928a-ef444ee7c1b1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:04.902188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 614/1205] 10.222.0.82 () {60 vars in 1132 bytes} [Wed Aug 24 03:34:04 2022] POST /identity/v3/users => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:34:06.882665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5189c9-1fcb-45d2-ace1-9bb6d3dcc0d0 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:06.883212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5189c9-1fcb-45d2-ace1-9bb6d3dcc0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:06.883326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5189c9-1fcb-45d2-ace1-9bb6d3dcc0d0 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:06.884603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 592/1206] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:06.890136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8933688-3d29-48e7-a75c-af8a4b36e432 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:06.890555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8933688-3d29-48e7-a75c-af8a4b36e432 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:06.890732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8933688-3d29-48e7-a75c-af8a4b36e432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:06.909456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e8933688-3d29-48e7-a75c-af8a4b36e432 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:06.910695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e8933688-3d29-48e7-a75c-af8a4b36e432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:06.980162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 615/1207] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:07.252805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5189c9-1fcb-45d2-ace1-9bb6d3dcc0d0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:07.359419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5189c9-1fcb-45d2-ace1-9bb6d3dcc0d0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:07.396214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206edc26-e457-4aba-a771-5ea67b3073cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:07.396842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206edc26-e457-4aba-a771-5ea67b3073cc None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:07.397129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206edc26-e457-4aba-a771-5ea67b3073cc None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:07.397407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206edc26-e457-4aba-a771-5ea67b3073cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:07.409507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206edc26-e457-4aba-a771-5ea67b3073cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-206edc26-e457-4aba-a771-5ea67b3073cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:34:07.414292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:07.415563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:07.416447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 593/1208] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:34:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:07.422197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8933688-3d29-48e7-a75c-af8a4b36e432 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:07.463683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:07.526260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8933688-3d29-48e7-a75c-af8a4b36e432 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:07.560043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c00ec-11e7-4f98-93ef-f68be4f2553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:07.560654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c00ec-11e7-4f98-93ef-f68be4f2553a None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:07.560930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c00ec-11e7-4f98-93ef-f68be4f2553a None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:07.561204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c00ec-11e7-4f98-93ef-f68be4f2553a None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:07.561895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c00ec-11e7-4f98-93ef-f68be4f2553a None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:07.562225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c00ec-11e7-4f98-93ef-f68be4f2553a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:07.563669 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c00ec-11e7-4f98-93ef-f68be4f2553a None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:07.571909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 616/1209] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:34:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:07.577970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206edc26-e457-4aba-a771-5ea67b3073cc None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:07.632375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:07.681822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206edc26-e457-4aba-a771-5ea67b3073cc None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:07.717819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7665732-991c-4e50-a303-4606ae33b8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:07.718458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7665732-991c-4e50-a303-4606ae33b8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:07.718775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7665732-991c-4e50-a303-4606ae33b8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:07.719050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7665732-991c-4e50-a303-4606ae33b8d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:07.730589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7665732-991c-4e50-a303-4606ae33b8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:07.732046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7665732-991c-4e50-a303-4606ae33b8d8 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-e7665732-991c-4e50-a303-4606ae33b8d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:07.750984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:07.752252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:07.753396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:07.753396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:07.753396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:07.754751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 594/1210] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:07.760845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c00ec-11e7-4f98-93ef-f68be4f2553a None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:07.859971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:07.862930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c00ec-11e7-4f98-93ef-f68be4f2553a None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:07.895863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c61b-818a-4e49-9c4a-05323ace87c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:07.896488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c61b-818a-4e49-9c4a-05323ace87c9 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:07.896730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c61b-818a-4e49-9c4a-05323ace87c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:07.897002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c61b-818a-4e49-9c4a-05323ace87c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:07.897644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8c61b-818a-4e49-9c4a-05323ace87c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:07.897943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8c61b-818a-4e49-9c4a-05323ace87c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:07.899335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8c61b-818a-4e49-9c4a-05323ace87c9 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:07.910836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 617/1211] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:07.916877 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7665732-991c-4e50-a303-4606ae33b8d8 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:08.018573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7665732-991c-4e50-a303-4606ae33b8d8 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:08.020639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:08.054385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155111c-2cd1-4be6-8f6e-b80650c7f2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:08.055039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155111c-2cd1-4be6-8f6e-b80650c7f2aa None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:08.055325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155111c-2cd1-4be6-8f6e-b80650c7f2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:08.055600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155111c-2cd1-4be6-8f6e-b80650c7f2aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:08.062469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155111c-2cd1-4be6-8f6e-b80650c7f2aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:08.065435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155111c-2cd1-4be6-8f6e-b80650c7f2aa None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-8155111c-2cd1-4be6-8f6e-b80650c7f2aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:34:08.076671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:08.077848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:08.079167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 595/1212] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:08.084969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c61b-818a-4e49-9c4a-05323ace87c9 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:08.188162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c61b-818a-4e49-9c4a-05323ace87c9 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:08.222330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00926ae3-c687-440b-8f57-9c78073a1f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:08.222968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00926ae3-c687-440b-8f57-9c78073a1f6c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:08.223248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00926ae3-c687-440b-8f57-9c78073a1f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:08.223519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00926ae3-c687-440b-8f57-9c78073a1f6c None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:08.224247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00926ae3-c687-440b-8f57-9c78073a1f6c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:08.224554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00926ae3-c687-440b-8f57-9c78073a1f6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:08.225987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00926ae3-c687-440b-8f57-9c78073a1f6c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:08.243817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 618/1213] 10.222.0.82 () {56 vars in 1143 bytes} [Wed Aug 24 03:34:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:08.249982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155111c-2cd1-4be6-8f6e-b80650c7f2aa None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:08.349763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155111c-2cd1-4be6-8f6e-b80650c7f2aa None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:08.384005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e1301-3867-4a7e-b59e-40205c2affd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:08.384643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e1301-3867-4a7e-b59e-40205c2affd2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:08.384932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e1301-3867-4a7e-b59e-40205c2affd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:08.385213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e1301-3867-4a7e-b59e-40205c2affd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:08.416555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:08.456258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e1301-3867-4a7e-b59e-40205c2affd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:08.457672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e1301-3867-4a7e-b59e-40205c2affd2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:08.556218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 596/1214] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:34:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:34:08.562110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00926ae3-c687-440b-8f57-9c78073a1f6c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:08.581620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:08.660860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00926ae3-c687-440b-8f57-9c78073a1f6c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:08.695291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099e7a-bcf5-460d-bb4c-c70cc9ca4451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:08.695921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099e7a-bcf5-460d-bb4c-c70cc9ca4451 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:08.696200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099e7a-bcf5-460d-bb4c-c70cc9ca4451 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:08.696473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099e7a-bcf5-460d-bb4c-c70cc9ca4451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:08.706656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37099e7a-bcf5-460d-bb4c-c70cc9ca4451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:08.708048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37099e7a-bcf5-460d-bb4c-c70cc9ca4451 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-37099e7a-bcf5-460d-bb4c-c70cc9ca4451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:08.722158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:08.723451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:08.724434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:08.724434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:08.724434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:08.724434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 619/1215] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:08.729097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e1301-3867-4a7e-b59e-40205c2affd2 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:08.791510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:08.829143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e1301-3867-4a7e-b59e-40205c2affd2 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:08.863299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab62faa-5483-4fad-b984-fd6dd954150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:08.863946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab62faa-5483-4fad-b984-fd6dd954150b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:08.864229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab62faa-5483-4fad-b984-fd6dd954150b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:08.864505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab62faa-5483-4fad-b984-fd6dd954150b None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:08.865151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab62faa-5483-4fad-b984-fd6dd954150b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:08.865457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab62faa-5483-4fad-b984-fd6dd954150b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:08.866862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab62faa-5483-4fad-b984-fd6dd954150b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:08.879150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 597/1216] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:08.884990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099e7a-bcf5-460d-bb4c-c70cc9ca4451 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:08.953631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:08.985533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099e7a-bcf5-460d-bb4c-c70cc9ca4451 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:09.021075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c686c4-51ab-411d-810d-aa24e3166fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:09.021075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c686c4-51ab-411d-810d-aa24e3166fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:09.023404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c686c4-51ab-411d-810d-aa24e3166fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:09.023404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c686c4-51ab-411d-810d-aa24e3166fc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:09.028301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c686c4-51ab-411d-810d-aa24e3166fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:09.029739 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c686c4-51ab-411d-810d-aa24e3166fc1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-41c686c4-51ab-411d-810d-aa24e3166fc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:34:09.041455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:09.042488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:09.043454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:34:09.043454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:34:09.043454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:34:09.043454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:34:09.043454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:34:09.043454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:34:09.043454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:09.043454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:09.043454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 620/1217] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:34:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:09.049168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab62faa-5483-4fad-b984-fd6dd954150b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:09.152286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab62faa-5483-4fad-b984-fd6dd954150b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:09.176253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:09.187962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57853967-d95f-4945-8818-f344e2a7e56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:09.188573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57853967-d95f-4945-8818-f344e2a7e56f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:09.188916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57853967-d95f-4945-8818-f344e2a7e56f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:09.189810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57853967-d95f-4945-8818-f344e2a7e56f None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:09.190527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57853967-d95f-4945-8818-f344e2a7e56f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:09.190853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57853967-d95f-4945-8818-f344e2a7e56f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:09.192309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57853967-d95f-4945-8818-f344e2a7e56f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:09.203770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 598/1218] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:34:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:09.210041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c686c4-51ab-411d-810d-aa24e3166fc1 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:09.309976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c686c4-51ab-411d-810d-aa24e3166fc1 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:09.338783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:09.344494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da8944-dda4-4e41-9952-f2e8870a16ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:09.345108 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da8944-dda4-4e41-9952-f2e8870a16ea None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:09.345432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da8944-dda4-4e41-9952-f2e8870a16ea None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:09.345730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da8944-dda4-4e41-9952-f2e8870a16ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:09.346448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da8944-dda4-4e41-9952-f2e8870a16ea None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=cff23ae056ff452588cd0b6520e96962) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:09.346800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da8944-dda4-4e41-9952-f2e8870a16ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:09.348249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da8944-dda4-4e41-9952-f2e8870a16ea None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:09.353644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 621/1219] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:34:09 2022] GET /identity/v3/role_assignments?user.id=cff23ae056ff452588cd0b6520e96962&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:11.255351 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d27461-0594-49e1-9afb-59e05e16fd51 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:11.255946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d27461-0594-49e1-9afb-59e05e16fd51 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:11.256163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d27461-0594-49e1-9afb-59e05e16fd51 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:11.257780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 599/1220] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:11.263122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683f5d-91e3-472b-b0e0-6d30f26a0d4c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:11.267700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683f5d-91e3-472b-b0e0-6d30f26a0d4c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:11.267700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683f5d-91e3-472b-b0e0-6d30f26a0d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:11.282843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6b683f5d-91e3-472b-b0e0-6d30f26a0d4c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:11.284025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6b683f5d-91e3-472b-b0e0-6d30f26a0d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:11.350110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 622/1221] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:11.616278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d27461-0594-49e1-9afb-59e05e16fd51 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:11.718733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d27461-0594-49e1-9afb-59e05e16fd51 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:11.754529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6d19b-9fc4-43fc-a7d2-35313b844428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:11.755176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6d19b-9fc4-43fc-a7d2-35313b844428 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:11.755462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6d19b-9fc4-43fc-a7d2-35313b844428 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:11.755741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6d19b-9fc4-43fc-a7d2-35313b844428 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:11.766868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6d19b-9fc4-43fc-a7d2-35313b844428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-52a6d19b-9fc4-43fc-a7d2-35313b844428 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:34:11.772142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:11.773342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:11.774312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 600/1222] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:34:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:11.779307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b683f5d-91e3-472b-b0e0-6d30f26a0d4c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:11.891085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b683f5d-91e3-472b-b0e0-6d30f26a0d4c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:11.916152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:11.927196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb057e3-ddc5-4a0a-9603-6e393f4bd670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:11.927814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb057e3-ddc5-4a0a-9603-6e393f4bd670 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:11.928092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb057e3-ddc5-4a0a-9603-6e393f4bd670 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:11.928364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb057e3-ddc5-4a0a-9603-6e393f4bd670 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:11.929032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb057e3-ddc5-4a0a-9603-6e393f4bd670 None admin] RBAC: Adding query filter params (name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:11.929335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb057e3-ddc5-4a0a-9603-6e393f4bd670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:11.930748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb057e3-ddc5-4a0a-9603-6e393f4bd670 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:11.940006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 623/1223] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:34:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:11.945863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6d19b-9fc4-43fc-a7d2-35313b844428 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:11.992217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:12.052760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6d19b-9fc4-43fc-a7d2-35313b844428 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:12.088976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7abb92-c871-43fa-ab96-9ab7a8e94a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:12.089720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7abb92-c871-43fa-ab96-9ab7a8e94a80 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:12.090009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7abb92-c871-43fa-ab96-9ab7a8e94a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:12.090286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7abb92-c871-43fa-ab96-9ab7a8e94a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:12.101644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7abb92-c871-43fa-ab96-9ab7a8e94a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:12.103158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7abb92-c871-43fa-ab96-9ab7a8e94a80 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-7e7abb92-c871-43fa-ab96-9ab7a8e94a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:12.119033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:12.120247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:12.121183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:12.121183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:12.121183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:12.121183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 601/1224] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:12.126370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb057e3-ddc5-4a0a-9603-6e393f4bd670 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:12.234673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb057e3-ddc5-4a0a-9603-6e393f4bd670 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:12.271118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd27a06-dac1-43ec-8edf-2d3b446a5060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:12.271972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd27a06-dac1-43ec-8edf-2d3b446a5060 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:12.272069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd27a06-dac1-43ec-8edf-2d3b446a5060 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:12.272268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd27a06-dac1-43ec-8edf-2d3b446a5060 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:12.272918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd27a06-dac1-43ec-8edf-2d3b446a5060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:12.273225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd27a06-dac1-43ec-8edf-2d3b446a5060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:12.274634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd27a06-dac1-43ec-8edf-2d3b446a5060 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:12.287399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 624/1225] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:12.293368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7abb92-c871-43fa-ab96-9ab7a8e94a80 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:12.395192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:12.402318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7abb92-c871-43fa-ab96-9ab7a8e94a80 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:12.440744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5c4d1-03b5-4d1b-93f0-719f74f2412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:12.443176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5c4d1-03b5-4d1b-93f0-719f74f2412d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:12.443176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5c4d1-03b5-4d1b-93f0-719f74f2412d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:12.443176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5c4d1-03b5-4d1b-93f0-719f74f2412d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:12.449462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5c4d1-03b5-4d1b-93f0-719f74f2412d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:12.450958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5c4d1-03b5-4d1b-93f0-719f74f2412d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-64a5c4d1-03b5-4d1b-93f0-719f74f2412d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:34:12.462799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:12.464050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:12.465232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 602/1226] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:12.470043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd27a06-dac1-43ec-8edf-2d3b446a5060 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:12.520473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:12.583532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd27a06-dac1-43ec-8edf-2d3b446a5060 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:12.620216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9d889-a878-4f06-8bce-bdd951eabc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:12.620855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9d889-a878-4f06-8bce-bdd951eabc5b None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:12.621134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9d889-a878-4f06-8bce-bdd951eabc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:12.621404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9d889-a878-4f06-8bce-bdd951eabc5b None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:12.622139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9d889-a878-4f06-8bce-bdd951eabc5b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:12.622733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9d889-a878-4f06-8bce-bdd951eabc5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:12.624173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9d889-a878-4f06-8bce-bdd951eabc5b None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:12.643720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 625/1227] 10.222.0.82 () {56 vars in 1143 bytes} [Wed Aug 24 03:34:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:12.649494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5c4d1-03b5-4d1b-93f0-719f74f2412d None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:12.758416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5c4d1-03b5-4d1b-93f0-719f74f2412d None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:12.794379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4b918-2c21-407f-a23b-808101e6f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:12.794963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4b918-2c21-407f-a23b-808101e6f02b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:12.795243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4b918-2c21-407f-a23b-808101e6f02b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:12.795516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4b918-2c21-407f-a23b-808101e6f02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:12.806716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4b918-2c21-407f-a23b-808101e6f02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:12.808182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4b918-2c21-407f-a23b-808101e6f02b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-a3a4b918-2c21-407f-a23b-808101e6f02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:12.823466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:12.824489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:12.825485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:12.825485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:12.825485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:12.825485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 603/1228] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:12.831624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9d889-a878-4f06-8bce-bdd951eabc5b None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:12.942441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9d889-a878-4f06-8bce-bdd951eabc5b None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:12.944888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:12.978940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b65ce5-2c32-4e1b-b0d7-abbe434e7d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:12.979562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b65ce5-2c32-4e1b-b0d7-abbe434e7d45 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:12.979945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b65ce5-2c32-4e1b-b0d7-abbe434e7d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:12.980124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b65ce5-2c32-4e1b-b0d7-abbe434e7d45 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:12.980780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b65ce5-2c32-4e1b-b0d7-abbe434e7d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:12.981088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b65ce5-2c32-4e1b-b0d7-abbe434e7d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:12.982473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b65ce5-2c32-4e1b-b0d7-abbe434e7d45 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:12.996519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 626/1229] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:13.003449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4b918-2c21-407f-a23b-808101e6f02b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:13.013474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:13.123604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4b918-2c21-407f-a23b-808101e6f02b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:13.159378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c0b47-25f2-4acf-bdc2-c794ec7d1f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:13.160012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c0b47-25f2-4acf-bdc2-c794ec7d1f73 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:13.160299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c0b47-25f2-4acf-bdc2-c794ec7d1f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:13.160575 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c0b47-25f2-4acf-bdc2-c794ec7d1f73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:13.167402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c0b47-25f2-4acf-bdc2-c794ec7d1f73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:13.168871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c0b47-25f2-4acf-bdc2-c794ec7d1f73 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-912c0b47-25f2-4acf-bdc2-c794ec7d1f73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:34:13.181252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:13.182702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:13.183828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:34:13.183828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:34:13.183828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:34:13.183828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:34:13.183828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:34:13.183828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:34:13.183828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:13.183828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:13.183828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 604/1230] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:34:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:13.188747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b65ce5-2c32-4e1b-b0d7-abbe434e7d45 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:13.299534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b65ce5-2c32-4e1b-b0d7-abbe434e7d45 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:13.339607 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99c629-be8d-46b0-b0cb-37804fd8a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:13.342139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99c629-be8d-46b0-b0cb-37804fd8a9fc None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:13.342139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99c629-be8d-46b0-b0cb-37804fd8a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:13.342139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99c629-be8d-46b0-b0cb-37804fd8a9fc None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:13.342139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99c629-be8d-46b0-b0cb-37804fd8a9fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:13.342139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99c629-be8d-46b0-b0cb-37804fd8a9fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:13.343220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99c629-be8d-46b0-b0cb-37804fd8a9fc None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:13.356415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 627/1231] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:34:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:13.362823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c0b47-25f2-4acf-bdc2-c794ec7d1f73 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:13.433815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:13.472595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c0b47-25f2-4acf-bdc2-c794ec7d1f73 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:13.512695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976d06c-2efa-4d51-a7c4-cc42800a69cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:13.513364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976d06c-2efa-4d51-a7c4-cc42800a69cd None admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:13.513648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976d06c-2efa-4d51-a7c4-cc42800a69cd None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:13.513925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976d06c-2efa-4d51-a7c4-cc42800a69cd None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/cff23ae056ff452588cd0b6520e96962/roles/9d16c3fb52ac40068ddddbf66411c519` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:13.530264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976d06c-2efa-4d51-a7c4-cc42800a69cd None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=cff23ae056ff452588cd0b6520e96962, role_id=9d16c3fb52ac40068ddddbf66411c519)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:13.531833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976d06c-2efa-4d51-a7c4-cc42800a69cd None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:13.686043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 605/1232] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:34:13 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/cff23ae056ff452588cd0b6520e96962/roles/9d16c3fb52ac40068ddddbf66411c519 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:34:15.613333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57cf61-7abd-49e7-95b9-5c880164b59f None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:15.613870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57cf61-7abd-49e7-95b9-5c880164b59f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:15.613870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57cf61-7abd-49e7-95b9-5c880164b59f None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:15.615312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 628/1233] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:15.620573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883b62a-d6ca-48af-ad0b-e124a41e4ce4 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:15.621015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883b62a-d6ca-48af-ad0b-e124a41e4ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:15.621165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883b62a-d6ca-48af-ad0b-e124a41e4ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:15.640038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e883b62a-d6ca-48af-ad0b-e124a41e4ce4 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:15.641248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e883b62a-d6ca-48af-ad0b-e124a41e4ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:15.711574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 606/1234] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:15.987967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57cf61-7abd-49e7-95b9-5c880164b59f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:16.014085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:16.095384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57cf61-7abd-49e7-95b9-5c880164b59f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:16.133655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77574-c408-47f8-a4be-d6273c334d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:16.134313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77574-c408-47f8-a4be-d6273c334d4e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:16.134615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77574-c408-47f8-a4be-d6273c334d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:16.134896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77574-c408-47f8-a4be-d6273c334d4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:16.147190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c77574-c408-47f8-a4be-d6273c334d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-91c77574-c408-47f8-a4be-d6273c334d4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:34:16.152557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:16.153674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:34:16.154780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:16.158701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 629/1235] 10.222.0.82 () {56 vars in 1102 bytes} [Wed Aug 24 03:34:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:16.160242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883b62a-d6ca-48af-ad0b-e124a41e4ce4 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:16.209539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:16.262609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883b62a-d6ca-48af-ad0b-e124a41e4ce4 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:16.301849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436f4a-7e9a-4dca-b4ef-7cb723d5056f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:16.302469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436f4a-7e9a-4dca-b4ef-7cb723d5056f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:16.302782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436f4a-7e9a-4dca-b4ef-7cb723d5056f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:16.303059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45436f4a-7e9a-4dca-b4ef-7cb723d5056f None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:16.303737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436f4a-7e9a-4dca-b4ef-7cb723d5056f None admin] RBAC: Adding query filter params (name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:16.304042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436f4a-7e9a-4dca-b4ef-7cb723d5056f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:16.305425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45436f4a-7e9a-4dca-b4ef-7cb723d5056f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:16.313789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 607/1236] 10.222.0.82 () {56 vars in 1103 bytes} [Wed Aug 24 03:34:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:16.321657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77574-c408-47f8-a4be-d6273c334d4e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:16.416176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:16.434029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77574-c408-47f8-a4be-d6273c334d4e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:16.469745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59527ef2-47b2-4648-a10c-b0aa0b66ec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:16.471143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59527ef2-47b2-4648-a10c-b0aa0b66ec6f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:16.471494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59527ef2-47b2-4648-a10c-b0aa0b66ec6f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:16.471782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59527ef2-47b2-4648-a10c-b0aa0b66ec6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:16.482471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59527ef2-47b2-4648-a10c-b0aa0b66ec6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:16.484022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59527ef2-47b2-4648-a10c-b0aa0b66ec6f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-59527ef2-47b2-4648-a10c-b0aa0b66ec6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:16.503855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:16.504941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:16.505979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:16.505979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:16.505979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:16.505979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 630/1237] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:16.511315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436f4a-7e9a-4dca-b4ef-7cb723d5056f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:16.615774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45436f4a-7e9a-4dca-b4ef-7cb723d5056f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:16.624599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:16.648996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725289bc-6898-4c3a-86aa-cf904b29ff66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:16.649596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725289bc-6898-4c3a-86aa-cf904b29ff66 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:16.649879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725289bc-6898-4c3a-86aa-cf904b29ff66 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:16.650152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725289bc-6898-4c3a-86aa-cf904b29ff66 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:16.650822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725289bc-6898-4c3a-86aa-cf904b29ff66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:16.651122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725289bc-6898-4c3a-86aa-cf904b29ff66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:16.652497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725289bc-6898-4c3a-86aa-cf904b29ff66 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:16.664097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 608/1238] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:16.669931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59527ef2-47b2-4648-a10c-b0aa0b66ec6f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:16.771546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59527ef2-47b2-4648-a10c-b0aa0b66ec6f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:16.817230 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f6d80-8f53-4236-9a3f-a2ec9446f6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:16.817974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f6d80-8f53-4236-9a3f-a2ec9446f6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:16.818303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f6d80-8f53-4236-9a3f-a2ec9446f6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:16.818668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f6d80-8f53-4236-9a3f-a2ec9446f6f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:16.821633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:16.827551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f6d80-8f53-4236-9a3f-a2ec9446f6f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:16.829001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f6d80-8f53-4236-9a3f-a2ec9446f6f7 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-7a9f6d80-8f53-4236-9a3f-a2ec9446f6f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:34:16.840938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:16.847383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:16.849145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 631/1239] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:16.850217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725289bc-6898-4c3a-86aa-cf904b29ff66 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:16.953838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725289bc-6898-4c3a-86aa-cf904b29ff66 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:16.987205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23357c1a-24d0-4bf8-949f-a14c19d51239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:16.987801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23357c1a-24d0-4bf8-949f-a14c19d51239 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:16.988079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23357c1a-24d0-4bf8-949f-a14c19d51239 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:16.988346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23357c1a-24d0-4bf8-949f-a14c19d51239 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:16.989064 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23357c1a-24d0-4bf8-949f-a14c19d51239 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:16.989368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23357c1a-24d0-4bf8-949f-a14c19d51239 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:16.990819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23357c1a-24d0-4bf8-949f-a14c19d51239 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:17.009724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 609/1240] 10.222.0.82 () {56 vars in 1143 bytes} [Wed Aug 24 03:34:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:17.015538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f6d80-8f53-4236-9a3f-a2ec9446f6f7 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:17.116653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f6d80-8f53-4236-9a3f-a2ec9446f6f7 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:17.151882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b97c7e-4adf-45de-bc9b-17a7fa6794a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:17.152554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b97c7e-4adf-45de-bc9b-17a7fa6794a5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:17.152838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b97c7e-4adf-45de-bc9b-17a7fa6794a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:17.153113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b97c7e-4adf-45de-bc9b-17a7fa6794a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:17.217773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:17.225228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b97c7e-4adf-45de-bc9b-17a7fa6794a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:17.226720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b97c7e-4adf-45de-bc9b-17a7fa6794a5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:17.325022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 632/1241] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:34:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:34:17.331167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23357c1a-24d0-4bf8-949f-a14c19d51239 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:17.359737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:17.432390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23357c1a-24d0-4bf8-949f-a14c19d51239 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:17.466072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5bb08-a17c-415c-9724-dc602371780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:17.466723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5bb08-a17c-415c-9724-dc602371780b None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:17.467004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5bb08-a17c-415c-9724-dc602371780b None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:17.467274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5bb08-a17c-415c-9724-dc602371780b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:17.477868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5bb08-a17c-415c-9724-dc602371780b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:17.479284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5bb08-a17c-415c-9724-dc602371780b None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-e4d5bb08-a17c-415c-9724-dc602371780b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:17.493865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:17.495008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:17.496341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:17.496341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:17.496341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:17.496341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 610/1242] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:17.501057 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b97c7e-4adf-45de-bc9b-17a7fa6794a5 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:17.609152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:17.614980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b97c7e-4adf-45de-bc9b-17a7fa6794a5 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:17.648153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb300fa-3b52-400a-a7de-010941547154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:17.648762 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb300fa-3b52-400a-a7de-010941547154 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:17.649064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb300fa-3b52-400a-a7de-010941547154 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:17.649339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb300fa-3b52-400a-a7de-010941547154 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:17.650046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb300fa-3b52-400a-a7de-010941547154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:17.650347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb300fa-3b52-400a-a7de-010941547154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:17.651747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb300fa-3b52-400a-a7de-010941547154 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:17.663092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 633/1243] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:17.668960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5bb08-a17c-415c-9724-dc602371780b None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:17.755146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:17.776921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5bb08-a17c-415c-9724-dc602371780b None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:17.810288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1aa0c-cc67-4e77-8f5c-ccb8ed57615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:17.810949 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1aa0c-cc67-4e77-8f5c-ccb8ed57615f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:17.811235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1aa0c-cc67-4e77-8f5c-ccb8ed57615f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:17.811511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1aa0c-cc67-4e77-8f5c-ccb8ed57615f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:17.818054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1aa0c-cc67-4e77-8f5c-ccb8ed57615f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:17.819544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1aa0c-cc67-4e77-8f5c-ccb8ed57615f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-0ad1aa0c-cc67-4e77-8f5c-ccb8ed57615f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:34:17.830565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:17.831680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:17.837610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:34:17.837610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:34:17.837610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:34:17.837610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:34:17.837610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:34:17.837610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:34:17.837610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:17.837610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:17.837610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 611/1244] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:34:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:17.838219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb300fa-3b52-400a-a7de-010941547154 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:17.942612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb300fa-3b52-400a-a7de-010941547154 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:17.976819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae31d0b-89a5-402b-93c1-2b855f5b4d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:17.977426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae31d0b-89a5-402b-93c1-2b855f5b4d3c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:17.977708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae31d0b-89a5-402b-93c1-2b855f5b4d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:17.977984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae31d0b-89a5-402b-93c1-2b855f5b4d3c None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:17.978706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae31d0b-89a5-402b-93c1-2b855f5b4d3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:17.979019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae31d0b-89a5-402b-93c1-2b855f5b4d3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:17.980457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae31d0b-89a5-402b-93c1-2b855f5b4d3c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:17.993115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 634/1245] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:34:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:18.015788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1aa0c-cc67-4e77-8f5c-ccb8ed57615f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:18.101407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1aa0c-cc67-4e77-8f5c-ccb8ed57615f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:18.135130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c9276-f8af-4479-8513-bf51aa8c815d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:18.135722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c9276-f8af-4479-8513-bf51aa8c815d None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:18.136002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c9276-f8af-4479-8513-bf51aa8c815d None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:18.136273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c9276-f8af-4479-8513-bf51aa8c815d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:18.136995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c9276-f8af-4479-8513-bf51aa8c815d None admin] RBAC: Adding query filter params (role.id=9d16c3fb52ac40068ddddbf66411c519, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=cff23ae056ff452588cd0b6520e96962) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:18.137303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c9276-f8af-4479-8513-bf51aa8c815d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:18.138749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c9276-f8af-4479-8513-bf51aa8c815d None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:18.144805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 612/1246] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:34:17 2022] GET /identity/v3/role_assignments?user.id=cff23ae056ff452588cd0b6520e96962&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9d16c3fb52ac40068ddddbf66411c519 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:20.155951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b72c1-ed0e-42b0-9b5f-0ee082e3b718 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:20.156502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b72c1-ed0e-42b0-9b5f-0ee082e3b718 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:20.156502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b72c1-ed0e-42b0-9b5f-0ee082e3b718 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:20.158262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 635/1247] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:20.163744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483f6e7-f459-498b-9378-4ff0e94fc9ee None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:20.164204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483f6e7-f459-498b-9378-4ff0e94fc9ee None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:20.164306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483f6e7-f459-498b-9378-4ff0e94fc9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:20.170750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:20.182814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f483f6e7-f459-498b-9378-4ff0e94fc9ee None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:20.184010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f483f6e7-f459-498b-9378-4ff0e94fc9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:20.249097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 613/1248] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:20.521192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b72c1-ed0e-42b0-9b5f-0ee082e3b718 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:20.540520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:20.623727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b72c1-ed0e-42b0-9b5f-0ee082e3b718 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:20.658862 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef529b4-fa87-4051-b662-e5601a3946fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:20.659483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef529b4-fa87-4051-b662-e5601a3946fc None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:20.659765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef529b4-fa87-4051-b662-e5601a3946fc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:20.660036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef529b4-fa87-4051-b662-e5601a3946fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:20.669733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef529b4-fa87-4051-b662-e5601a3946fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-0ef529b4-fa87-4051-b662-e5601a3946fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:34:20.674761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:20.678593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:20.679716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 636/1249] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:34:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:20.681872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483f6e7-f459-498b-9378-4ff0e94fc9ee None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:20.779442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483f6e7-f459-498b-9378-4ff0e94fc9ee None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:20.811116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:20.816662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795e4cc-bfb5-4279-83c1-b9251d50ad0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:20.817270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795e4cc-bfb5-4279-83c1-b9251d50ad0e None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:20.817556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795e4cc-bfb5-4279-83c1-b9251d50ad0e None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:20.817834 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795e4cc-bfb5-4279-83c1-b9251d50ad0e None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:20.818500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795e4cc-bfb5-4279-83c1-b9251d50ad0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:20.818834 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795e4cc-bfb5-4279-83c1-b9251d50ad0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:20.820214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795e4cc-bfb5-4279-83c1-b9251d50ad0e None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:20.828114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 614/1250] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:34:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:20.834758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef529b4-fa87-4051-b662-e5601a3946fc None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:20.919281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:20.944940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef529b4-fa87-4051-b662-e5601a3946fc None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:20.979328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ab056-6920-4c54-a457-52033a606cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:20.979958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab056-6920-4c54-a457-52033a606cfa None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:20.980244 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab056-6920-4c54-a457-52033a606cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:20.980522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab056-6920-4c54-a457-52033a606cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:20.991628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ab056-6920-4c54-a457-52033a606cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:20.992687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ab056-6920-4c54-a457-52033a606cfa None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-d60ab056-6920-4c54-a457-52033a606cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:21.013285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:21.014552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:21.015731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:21.015731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:21.015731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:21.018702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 637/1251] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:21.022198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795e4cc-bfb5-4279-83c1-b9251d50ad0e None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:21.124848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795e4cc-bfb5-4279-83c1-b9251d50ad0e None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:21.158381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d906abd-3bde-4559-83fc-93f64dbbfa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:21.159017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d906abd-3bde-4559-83fc-93f64dbbfa01 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:21.159291 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d906abd-3bde-4559-83fc-93f64dbbfa01 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:21.159571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d906abd-3bde-4559-83fc-93f64dbbfa01 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:21.160222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d906abd-3bde-4559-83fc-93f64dbbfa01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:21.160523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d906abd-3bde-4559-83fc-93f64dbbfa01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:21.161905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d906abd-3bde-4559-83fc-93f64dbbfa01 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:21.174611 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 615/1252] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:21.180501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ab056-6920-4c54-a457-52033a606cfa None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:21.282668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ab056-6920-4c54-a457-52033a606cfa None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:21.308523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:21.316683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbb92e-48cd-426f-9f1a-5ec16b98d97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:21.317320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbb92e-48cd-426f-9f1a-5ec16b98d97c None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:21.317601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbb92e-48cd-426f-9f1a-5ec16b98d97c None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:21.317875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbb92e-48cd-426f-9f1a-5ec16b98d97c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:21.325130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbb92e-48cd-426f-9f1a-5ec16b98d97c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:21.326607 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbb92e-48cd-426f-9f1a-5ec16b98d97c None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-bffbb92e-48cd-426f-9f1a-5ec16b98d97c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:34:21.337494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:21.338483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:21.339813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 638/1253] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:21.344786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d906abd-3bde-4559-83fc-93f64dbbfa01 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:21.402034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:21.455794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d906abd-3bde-4559-83fc-93f64dbbfa01 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:21.505900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e95b9-d8c9-4abc-a022-ba670d7892c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:21.506549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e95b9-d8c9-4abc-a022-ba670d7892c7 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:21.506832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e95b9-d8c9-4abc-a022-ba670d7892c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:21.507109 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e95b9-d8c9-4abc-a022-ba670d7892c7 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:21.507920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e95b9-d8c9-4abc-a022-ba670d7892c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:21.508477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e95b9-d8c9-4abc-a022-ba670d7892c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:21.510235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e95b9-d8c9-4abc-a022-ba670d7892c7 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:21.537579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 616/1254] 10.222.0.82 () {56 vars in 1143 bytes} [Wed Aug 24 03:34:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:21.543524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbb92e-48cd-426f-9f1a-5ec16b98d97c None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:21.647696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbb92e-48cd-426f-9f1a-5ec16b98d97c None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:21.681488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2cdab-8cf9-4bb1-bb47-518ef8525065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:21.682092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2cdab-8cf9-4bb1-bb47-518ef8525065 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:21.682369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2cdab-8cf9-4bb1-bb47-518ef8525065 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:21.682670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2cdab-8cf9-4bb1-bb47-518ef8525065 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:21.739321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:21.756954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2cdab-8cf9-4bb1-bb47-518ef8525065 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:21.758411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2cdab-8cf9-4bb1-bb47-518ef8525065 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:21.865081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 639/1255] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:34:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:34:21.871079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e95b9-d8c9-4abc-a022-ba670d7892c7 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:21.994437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e95b9-d8c9-4abc-a022-ba670d7892c7 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:22.008262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:22.030394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f576b8-00e3-4849-8e8b-ab1d0db1fc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:22.031060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f576b8-00e3-4849-8e8b-ab1d0db1fc33 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:22.031339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f576b8-00e3-4849-8e8b-ab1d0db1fc33 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:22.031610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f576b8-00e3-4849-8e8b-ab1d0db1fc33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:22.043505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f576b8-00e3-4849-8e8b-ab1d0db1fc33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:22.044918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f576b8-00e3-4849-8e8b-ab1d0db1fc33 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-39f576b8-00e3-4849-8e8b-ab1d0db1fc33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:22.060241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:22.061519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:22.062683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:22.062683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:22.062683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:22.062683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 617/1256] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:22.068566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2cdab-8cf9-4bb1-bb47-518ef8525065 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:22.159146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:22.170801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2cdab-8cf9-4bb1-bb47-518ef8525065 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:22.207190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8336ca-5a98-431a-8bd5-c0ff4ec2bcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:22.207864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8336ca-5a98-431a-8bd5-c0ff4ec2bcdd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:22.208074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8336ca-5a98-431a-8bd5-c0ff4ec2bcdd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:22.208352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8336ca-5a98-431a-8bd5-c0ff4ec2bcdd None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:22.209006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8336ca-5a98-431a-8bd5-c0ff4ec2bcdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:22.209403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8336ca-5a98-431a-8bd5-c0ff4ec2bcdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:22.210831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8336ca-5a98-431a-8bd5-c0ff4ec2bcdd None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:22.224659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 640/1257] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:22.229600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f576b8-00e3-4849-8e8b-ab1d0db1fc33 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:22.332229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f576b8-00e3-4849-8e8b-ab1d0db1fc33 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:22.364706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ab811-713b-4bd0-a624-3ece220589ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:22.365328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ab811-713b-4bd0-a624-3ece220589ea None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:22.365608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ab811-713b-4bd0-a624-3ece220589ea None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:22.365879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ab811-713b-4bd0-a624-3ece220589ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:22.372318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ab811-713b-4bd0-a624-3ece220589ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:22.374011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ab811-713b-4bd0-a624-3ece220589ea None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-f27ab811-713b-4bd0-a624-3ece220589ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:34:22.386766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:22.387903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:22.388998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:34:22.388998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:34:22.388998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:34:22.388998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:34:22.388998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:34:22.388998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:34:22.388998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:22.388998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:22.388998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 618/1258] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:34:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:34:22.395270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8336ca-5a98-431a-8bd5-c0ff4ec2bcdd None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:22.503060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8336ca-5a98-431a-8bd5-c0ff4ec2bcdd None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:22.537846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d36c8-aaaa-4a46-addf-c1052a67d2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:22.538463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d36c8-aaaa-4a46-addf-c1052a67d2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:22.538779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d36c8-aaaa-4a46-addf-c1052a67d2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:22.539062 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d36c8-aaaa-4a46-addf-c1052a67d2b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:22.539758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d36c8-aaaa-4a46-addf-c1052a67d2b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:22.540069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d36c8-aaaa-4a46-addf-c1052a67d2b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:22.541497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d36c8-aaaa-4a46-addf-c1052a67d2b1 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:22.553476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 641/1259] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:34:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:22.559975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ab811-713b-4bd0-a624-3ece220589ea None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:22.587344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:22.662245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ab811-713b-4bd0-a624-3ece220589ea None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:22.699179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937432e-7f96-426f-afee-b0dc429f99ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:22.699781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937432e-7f96-426f-afee-b0dc429f99ad None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:22.700060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937432e-7f96-426f-afee-b0dc429f99ad None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:22.700330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937432e-7f96-426f-afee-b0dc429f99ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:22.701098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937432e-7f96-426f-afee-b0dc429f99ad None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=cff23ae056ff452588cd0b6520e96962) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:22.701405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937432e-7f96-426f-afee-b0dc429f99ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:22.702858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937432e-7f96-426f-afee-b0dc429f99ad None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:22.708759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 619/1260] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:34:22 2022] GET /identity/v3/role_assignments?user.id=cff23ae056ff452588cd0b6520e96962&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9e904216086e4a06abf9216b0c863f89 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:24.601889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a7d44-657d-4296-a12c-cf2283f41bfb None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:24.602450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a7d44-657d-4296-a12c-cf2283f41bfb None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:24.602450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a7d44-657d-4296-a12c-cf2283f41bfb None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:24.603820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 642/1261] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:24.609728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c10149-7255-4168-81c9-9abe6900d002 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:24.618596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c10149-7255-4168-81c9-9abe6900d002 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:24.618596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c10149-7255-4168-81c9-9abe6900d002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:24.630526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-90c10149-7255-4168-81c9-9abe6900d002 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:24.631844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-90c10149-7255-4168-81c9-9abe6900d002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:24.701131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 620/1262] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:24.977257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a7d44-657d-4296-a12c-cf2283f41bfb None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:24.989141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:25.082275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a7d44-657d-4296-a12c-cf2283f41bfb None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:25.115662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5da812-c70a-4549-94fb-4369a38db1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:25.116284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5da812-c70a-4549-94fb-4369a38db1fc None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:25.116560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5da812-c70a-4549-94fb-4369a38db1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:25.116827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5da812-c70a-4549-94fb-4369a38db1fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:25.128736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5da812-c70a-4549-94fb-4369a38db1fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-ac5da812-c70a-4549-94fb-4369a38db1fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:34:25.133679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:25.134941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:25.135898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 643/1263] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:34:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:25.141036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c10149-7255-4168-81c9-9abe6900d002 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:25.161939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:25.254082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c10149-7255-4168-81c9-9abe6900d002 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:25.288176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62936b-dcc2-48d9-9e6e-200af6897bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:25.288781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62936b-dcc2-48d9-9e6e-200af6897bef None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:25.289065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62936b-dcc2-48d9-9e6e-200af6897bef None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:25.289338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62936b-dcc2-48d9-9e6e-200af6897bef None admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:25.290111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62936b-dcc2-48d9-9e6e-200af6897bef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:25.290497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62936b-dcc2-48d9-9e6e-200af6897bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:25.291913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62936b-dcc2-48d9-9e6e-200af6897bef None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:25.301073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 621/1264] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:34:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:25.307011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5da812-c70a-4549-94fb-4369a38db1fc None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:25.379391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:25.412499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5da812-c70a-4549-94fb-4369a38db1fc None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:25.446423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea882f5-58a7-44f5-aa79-4d8db0d1a551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:25.447077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea882f5-58a7-44f5-aa79-4d8db0d1a551 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:25.447439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea882f5-58a7-44f5-aa79-4d8db0d1a551 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:25.447726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea882f5-58a7-44f5-aa79-4d8db0d1a551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:25.457734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea882f5-58a7-44f5-aa79-4d8db0d1a551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:25.459235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea882f5-58a7-44f5-aa79-4d8db0d1a551 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-2ea882f5-58a7-44f5-aa79-4d8db0d1a551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:25.475486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:25.476979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:25.477955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:25.477955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:25.477955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:25.477955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 644/1265] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:25.482723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62936b-dcc2-48d9-9e6e-200af6897bef None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:25.580430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:25.600460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62936b-dcc2-48d9-9e6e-200af6897bef None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:25.635056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd4fbd-9bae-4958-ae51-8d83ab73bfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:25.635721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd4fbd-9bae-4958-ae51-8d83ab73bfda None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:25.640625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd4fbd-9bae-4958-ae51-8d83ab73bfda None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:25.640625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd4fbd-9bae-4958-ae51-8d83ab73bfda None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:25.640625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd4fbd-9bae-4958-ae51-8d83ab73bfda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:25.640625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd4fbd-9bae-4958-ae51-8d83ab73bfda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:25.640625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd4fbd-9bae-4958-ae51-8d83ab73bfda None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:25.651103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 622/1266] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:25.656855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea882f5-58a7-44f5-aa79-4d8db0d1a551 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:25.759686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea882f5-58a7-44f5-aa79-4d8db0d1a551 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:25.779501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:25.796407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a034d9-d41e-4995-8562-145bcad08834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:25.797033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a034d9-d41e-4995-8562-145bcad08834 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:25.797312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a034d9-d41e-4995-8562-145bcad08834 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:25.797582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a034d9-d41e-4995-8562-145bcad08834 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:25.805671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a034d9-d41e-4995-8562-145bcad08834 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:25.807236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a034d9-d41e-4995-8562-145bcad08834 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-26a034d9-d41e-4995-8562-145bcad08834 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:34:25.818707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:25.819897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:25.821238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 645/1267] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:25.825741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd4fbd-9bae-4958-ae51-8d83ab73bfda None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:25.932922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd4fbd-9bae-4958-ae51-8d83ab73bfda None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:25.966186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ab83b-6afa-4443-a65a-741e38ec7b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:25.966829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ab83b-6afa-4443-a65a-741e38ec7b93 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:25.967112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ab83b-6afa-4443-a65a-741e38ec7b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:25.967388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ab83b-6afa-4443-a65a-741e38ec7b93 None admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:25.968118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ab83b-6afa-4443-a65a-741e38ec7b93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:25.968423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ab83b-6afa-4443-a65a-741e38ec7b93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:25.969847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ab83b-6afa-4443-a65a-741e38ec7b93 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:25.980526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:25.989656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 623/1268] 10.222.0.82 () {56 vars in 1143 bytes} [Wed Aug 24 03:34:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:25.995708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a034d9-d41e-4995-8562-145bcad08834 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:26.109623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a034d9-d41e-4995-8562-145bcad08834 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:26.145158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd333a-2a7d-4dab-97fe-01859233a68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:26.145792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd333a-2a7d-4dab-97fe-01859233a68f None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:26.146072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd333a-2a7d-4dab-97fe-01859233a68f None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:26.146344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd333a-2a7d-4dab-97fe-01859233a68f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:26.156923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd333a-2a7d-4dab-97fe-01859233a68f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:26.159186 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd333a-2a7d-4dab-97fe-01859233a68f None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-51cd333a-2a7d-4dab-97fe-01859233a68f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:26.172982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:26.175631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:26.176947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:26.176947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:26.176947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:26.176947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 646/1269] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:26.180714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ab83b-6afa-4443-a65a-741e38ec7b93 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:26.281236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ab83b-6afa-4443-a65a-741e38ec7b93 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:26.315824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252388a3-770e-4440-856c-d335ffd2ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:26.316429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252388a3-770e-4440-856c-d335ffd2ff97 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:26.316713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252388a3-770e-4440-856c-d335ffd2ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:26.316989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252388a3-770e-4440-856c-d335ffd2ff97 None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:26.317745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252388a3-770e-4440-856c-d335ffd2ff97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:26.318049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252388a3-770e-4440-856c-d335ffd2ff97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:26.319481 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252388a3-770e-4440-856c-d335ffd2ff97 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:26.330765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 624/1270] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:26.336623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd333a-2a7d-4dab-97fe-01859233a68f None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:26.369207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:26.436874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd333a-2a7d-4dab-97fe-01859233a68f None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:26.470327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ea5af-d0d6-46ec-af77-20dd7c880121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:26.470989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ea5af-d0d6-46ec-af77-20dd7c880121 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:26.471277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ea5af-d0d6-46ec-af77-20dd7c880121 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:26.471554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ea5af-d0d6-46ec-af77-20dd7c880121 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:26.478072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ea5af-d0d6-46ec-af77-20dd7c880121 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:26.479559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ea5af-d0d6-46ec-af77-20dd7c880121 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-604ea5af-d0d6-46ec-af77-20dd7c880121 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:34:26.492973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:26.494219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:26.495222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:34:26.495222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:34:26.495222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:34:26.495222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:34:26.495222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:34:26.495222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:34:26.495222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:26.495222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:26.495222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 647/1271] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:26.500638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252388a3-770e-4440-856c-d335ffd2ff97 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:26.608299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:26.649974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252388a3-770e-4440-856c-d335ffd2ff97 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:26.683706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b739b96-f4d8-4794-bf4c-19a8b2eab45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:26.684315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739b96-f4d8-4794-bf4c-19a8b2eab45c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:26.684603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739b96-f4d8-4794-bf4c-19a8b2eab45c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:26.684882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739b96-f4d8-4794-bf4c-19a8b2eab45c None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:26.685577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739b96-f4d8-4794-bf4c-19a8b2eab45c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:26.685888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739b96-f4d8-4794-bf4c-19a8b2eab45c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:26.687355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739b96-f4d8-4794-bf4c-19a8b2eab45c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:26.699460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 625/1272] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:34:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:34:26.709120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ea5af-d0d6-46ec-af77-20dd7c880121 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:26.816469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:26.825016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ea5af-d0d6-46ec-af77-20dd7c880121 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:26.860853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53240386-9cfe-485e-9a68-6e96da5f2a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:26.861751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53240386-9cfe-485e-9a68-6e96da5f2a06 None admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:26.862103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53240386-9cfe-485e-9a68-6e96da5f2a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:26.862517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53240386-9cfe-485e-9a68-6e96da5f2a06 None admin] PATH_INFO: `/v3/projects/4a94ab421b414abb89ad09b813fce957/users/cff23ae056ff452588cd0b6520e96962/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:26.864468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53240386-9cfe-485e-9a68-6e96da5f2a06 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a94ab421b414abb89ad09b813fce957, user_id=cff23ae056ff452588cd0b6520e96962, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:26.865963 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53240386-9cfe-485e-9a68-6e96da5f2a06 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:26.979837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 648/1273] 10.222.0.82 () {58 vars in 1439 bytes} [Wed Aug 24 03:34:26 2022] PUT /identity/v3/projects/4a94ab421b414abb89ad09b813fce957/users/cff23ae056ff452588cd0b6520e96962/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:34:29.054725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0880e6-c524-4755-91ad-ef35ef8bbd37 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:29.055344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0880e6-c524-4755-91ad-ef35ef8bbd37 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:29.055344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0880e6-c524-4755-91ad-ef35ef8bbd37 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:29.056867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 626/1274] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:29.062791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99121d6-a2ae-48aa-8243-751ccd1f81ee None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:29.063535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99121d6-a2ae-48aa-8243-751ccd1f81ee None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:29.063750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99121d6-a2ae-48aa-8243-751ccd1f81ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:29.084726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e99121d6-a2ae-48aa-8243-751ccd1f81ee None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:29.085999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e99121d6-a2ae-48aa-8243-751ccd1f81ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:29.164879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 649/1275] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:29.530103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0880e6-c524-4755-91ad-ef35ef8bbd37 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:29.660881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0880e6-c524-4755-91ad-ef35ef8bbd37 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:29.683270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:29.701899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94216d7-2248-485c-8aa4-35b2a2c3c1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:29.702581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94216d7-2248-485c-8aa4-35b2a2c3c1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:29.702871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94216d7-2248-485c-8aa4-35b2a2c3c1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:29.703247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94216d7-2248-485c-8aa4-35b2a2c3c1b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:29.715853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94216d7-2248-485c-8aa4-35b2a2c3c1b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-f94216d7-2248-485c-8aa4-35b2a2c3c1b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:34:29.721994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:29.730324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:29.732152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 627/1276] 10.222.0.82 () {56 vars in 1096 bytes} [Wed Aug 24 03:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:29.733679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99121d6-a2ae-48aa-8243-751ccd1f81ee None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:29.825069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:29.845072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99121d6-a2ae-48aa-8243-751ccd1f81ee None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:29.883300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d4c7e-b68c-4423-8591-f47e2c3d3600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:29.883954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d4c7e-b68c-4423-8591-f47e2c3d3600 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:29.884235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d4c7e-b68c-4423-8591-f47e2c3d3600 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:29.884510 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d4c7e-b68c-4423-8591-f47e2c3d3600 None admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:29.885243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d4c7e-b68c-4423-8591-f47e2c3d3600 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:29.885547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d4c7e-b68c-4423-8591-f47e2c3d3600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:29.886983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d4c7e-b68c-4423-8591-f47e2c3d3600 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:29.896767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 650/1277] 10.222.0.82 () {56 vars in 1099 bytes} [Wed Aug 24 03:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:29.902861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94216d7-2248-485c-8aa4-35b2a2c3c1b9 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:30.031534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94216d7-2248-485c-8aa4-35b2a2c3c1b9 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:30.073415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357e929-6c49-4fda-98e4-8ce2a4e18e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:30.074223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357e929-6c49-4fda-98e4-8ce2a4e18e7a None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:30.074419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357e929-6c49-4fda-98e4-8ce2a4e18e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:30.074694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357e929-6c49-4fda-98e4-8ce2a4e18e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:30.087489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357e929-6c49-4fda-98e4-8ce2a4e18e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:30.089300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357e929-6c49-4fda-98e4-8ce2a4e18e7a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-2357e929-6c49-4fda-98e4-8ce2a4e18e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:30.105576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:30.107470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:30.109201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:30.109201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:30.109201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:30.109201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 628/1278] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:30.113378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d4c7e-b68c-4423-8591-f47e2c3d3600 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:30.219745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d4c7e-b68c-4423-8591-f47e2c3d3600 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:30.257572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45aef5-2e9b-42ce-bd38-e5996a4691e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:30.258219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45aef5-2e9b-42ce-bd38-e5996a4691e5 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:30.258532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45aef5-2e9b-42ce-bd38-e5996a4691e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:30.258814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45aef5-2e9b-42ce-bd38-e5996a4691e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:30.259481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45aef5-2e9b-42ce-bd38-e5996a4691e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:30.259786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45aef5-2e9b-42ce-bd38-e5996a4691e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:30.261179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45aef5-2e9b-42ce-bd38-e5996a4691e5 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:30.274911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 651/1279] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:34:30.281503 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357e929-6c49-4fda-98e4-8ce2a4e18e7a None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:30.315737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:30.401462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357e929-6c49-4fda-98e4-8ce2a4e18e7a None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:30.437498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d70be-7ad0-4960-8c00-eb9bd58c35f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:30.438136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d70be-7ad0-4960-8c00-eb9bd58c35f3 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:30.438443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d70be-7ad0-4960-8c00-eb9bd58c35f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:30.438771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d70be-7ad0-4960-8c00-eb9bd58c35f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:30.447515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d70be-7ad0-4960-8c00-eb9bd58c35f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:30.448997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d70be-7ad0-4960-8c00-eb9bd58c35f3 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-748d70be-7ad0-4960-8c00-eb9bd58c35f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:34:30.463144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:30.464360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:30.465456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 629/1280] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:30.471318 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45aef5-2e9b-42ce-bd38-e5996a4691e5 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:30.475344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:30.583178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45aef5-2e9b-42ce-bd38-e5996a4691e5 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:30.620880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c95bdf-104a-41e1-ade3-894f7da54bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:30.621506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c95bdf-104a-41e1-ade3-894f7da54bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:30.621887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c95bdf-104a-41e1-ade3-894f7da54bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:30.622169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c95bdf-104a-41e1-ade3-894f7da54bf0 None admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:30.622945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c95bdf-104a-41e1-ade3-894f7da54bf0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:30.623258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c95bdf-104a-41e1-ade3-894f7da54bf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:30.625144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c95bdf-104a-41e1-ade3-894f7da54bf0 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:30.645729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 652/1281] 10.222.0.82 () {56 vars in 1143 bytes} [Wed Aug 24 03:34:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:30.651817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d70be-7ad0-4960-8c00-eb9bd58c35f3 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:30.753606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:30.768118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d70be-7ad0-4960-8c00-eb9bd58c35f3 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:30.813767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d966f5-3097-452c-b583-921a213dcc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:30.815137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d966f5-3097-452c-b583-921a213dcc81 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:30.815412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d966f5-3097-452c-b583-921a213dcc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:30.815815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d966f5-3097-452c-b583-921a213dcc81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:30.896700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d966f5-3097-452c-b583-921a213dcc81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:30.898716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d966f5-3097-452c-b583-921a213dcc81 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:30.997492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:31.010467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 630/1282] 10.222.0.82 () {58 vars in 1282 bytes} [Wed Aug 24 03:34:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:34:31.017003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c95bdf-104a-41e1-ade3-894f7da54bf0 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:31.090624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:31.137430 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c95bdf-104a-41e1-ade3-894f7da54bf0 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:31.176054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91060af2-cd50-4974-96af-7549e17399ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:31.176866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91060af2-cd50-4974-96af-7549e17399ad None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:31.177147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91060af2-cd50-4974-96af-7549e17399ad None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:31.177425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91060af2-cd50-4974-96af-7549e17399ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:31.187624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91060af2-cd50-4974-96af-7549e17399ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:31.189178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91060af2-cd50-4974-96af-7549e17399ad None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-91060af2-cd50-4974-96af-7549e17399ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:34:31.203984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:31.205783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:34:31.211030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:34:31.211030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:34:31.211030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:31.211030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 653/1283] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:31.212137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d966f5-3097-452c-b583-921a213dcc81 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:31.332230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d966f5-3097-452c-b583-921a213dcc81 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:31.366628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909722e-1ecf-4983-9e0a-13ff022b289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:31.367244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909722e-1ecf-4983-9e0a-13ff022b289f None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:31.367527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909722e-1ecf-4983-9e0a-13ff022b289f None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:31.367802 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909722e-1ecf-4983-9e0a-13ff022b289f None admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:31.368447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909722e-1ecf-4983-9e0a-13ff022b289f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:31.368804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909722e-1ecf-4983-9e0a-13ff022b289f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:31.370136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909722e-1ecf-4983-9e0a-13ff022b289f None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:31.382224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 631/1284] 10.222.0.82 () {56 vars in 1109 bytes} [Wed Aug 24 03:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:34:31.388264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91060af2-cd50-4974-96af-7549e17399ad None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:31.508178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91060af2-cd50-4974-96af-7549e17399ad None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:31.524798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:31.545487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd64365-987c-4f25-92f2-ea3972998f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:31.546106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd64365-987c-4f25-92f2-ea3972998f4e None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:31.546389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd64365-987c-4f25-92f2-ea3972998f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:31.546697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd64365-987c-4f25-92f2-ea3972998f4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:31.554709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd64365-987c-4f25-92f2-ea3972998f4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:31.555262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd64365-987c-4f25-92f2-ea3972998f4e None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application [None req-1dd64365-987c-4f25-92f2-ea3972998f4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:34:31.567529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:31.568688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:31.569845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:34:31.569845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:34:31.569845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:34:31.569845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:34:31.569845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:34:31.569845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:34:31.569845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:34:31.569845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: ERROR keystone.server.flask.application Aug 24 03:34:31.569845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 654/1285] 10.222.0.82 () {56 vars in 1111 bytes} [Wed Aug 24 03:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:31.578207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909722e-1ecf-4983-9e0a-13ff022b289f None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:31.634685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:31.689142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909722e-1ecf-4983-9e0a-13ff022b289f None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:31.734647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503b3e8-0176-4ebe-804d-36867a699ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:31.735244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503b3e8-0176-4ebe-804d-36867a699ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:31.735525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503b3e8-0176-4ebe-804d-36867a699ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:31.735800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503b3e8-0176-4ebe-804d-36867a699ec2 None admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:31.737036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503b3e8-0176-4ebe-804d-36867a699ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:31.737457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503b3e8-0176-4ebe-804d-36867a699ec2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:31.738943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503b3e8-0176-4ebe-804d-36867a699ec2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:31.752224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 632/1286] 10.222.0.82 () {56 vars in 1148 bytes} [Wed Aug 24 03:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:31.759352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd64365-987c-4f25-92f2-ea3972998f4e None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:31.882391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd64365-987c-4f25-92f2-ea3972998f4e None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:31.920059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e90d5-991d-4d0e-84b4-9d584a335063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:31.920680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e90d5-991d-4d0e-84b4-9d584a335063 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:31.920964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e90d5-991d-4d0e-84b4-9d584a335063 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:31.921238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e90d5-991d-4d0e-84b4-9d584a335063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:31.925550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e90d5-991d-4d0e-84b4-9d584a335063 None admin] RBAC: Adding query filter params (role.id=9e904216086e4a06abf9216b0c863f89, scope.project.id=4a94ab421b414abb89ad09b813fce957, user.id=cff23ae056ff452588cd0b6520e96962) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:31.925550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e90d5-991d-4d0e-84b4-9d584a335063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:31.925550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e90d5-991d-4d0e-84b4-9d584a335063 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:31.930428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 655/1287] 10.222.0.82 () {56 vars in 1374 bytes} [Wed Aug 24 03:34:31 2022] GET /identity/v3/role_assignments?user.id=cff23ae056ff452588cd0b6520e96962&scope.project.id=4a94ab421b414abb89ad09b813fce957&role.id=9e904216086e4a06abf9216b0c863f89 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:34:34.079529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b44665-e560-400d-bffe-8fb7768c47e1 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:34.080626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b44665-e560-400d-bffe-8fb7768c47e1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:34.080626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b44665-e560-400d-bffe-8fb7768c47e1 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:34.082815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 633/1288] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:34.088923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c1830-814d-4f59-acbc-b5c1b1c59e0c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:34.089454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c1830-814d-4f59-acbc-b5c1b1c59e0c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:34.089454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c1830-814d-4f59-acbc-b5c1b1c59e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:34.108439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e56c1830-814d-4f59-acbc-b5c1b1c59e0c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:34.109643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e56c1830-814d-4f59-acbc-b5c1b1c59e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:34.141158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:34.179049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 656/1289] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:34:34.448259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b44665-e560-400d-bffe-8fb7768c47e1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:34.550187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b44665-e560-400d-bffe-8fb7768c47e1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:34.560679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:34.584050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda747c7-b531-4252-987c-46b5fa50a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:34.584680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda747c7-b531-4252-987c-46b5fa50a8fc None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:34.584963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda747c7-b531-4252-987c-46b5fa50a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:34.585239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda747c7-b531-4252-987c-46b5fa50a8fc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:34.585828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda747c7-b531-4252-987c-46b5fa50a8fc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:34.587229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda747c7-b531-4252-987c-46b5fa50a8fc None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-eda747c7-b531-4252-987c-46b5fa50a8fc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:34.595829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:34.597062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:34.597950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 634/1290] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:34:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:34.603096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c1830-814d-4f59-acbc-b5c1b1c59e0c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:34.703759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c1830-814d-4f59-acbc-b5c1b1c59e0c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:34.738400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6e2d2-f2c5-4377-9f67-63ee23121c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:34.739033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6e2d2-f2c5-4377-9f67-63ee23121c64 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:34.739310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6e2d2-f2c5-4377-9f67-63ee23121c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:34.739582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6e2d2-f2c5-4377-9f67-63ee23121c64 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:34.740206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6e2d2-f2c5-4377-9f67-63ee23121c64 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:34.740507 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6e2d2-f2c5-4377-9f67-63ee23121c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:34.741869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6e2d2-f2c5-4377-9f67-63ee23121c64 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:34.746662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 657/1291] 10.222.0.82 () {56 vars in 1116 bytes} [Wed Aug 24 03:34:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:34.753441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda747c7-b531-4252-987c-46b5fa50a8fc None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:34.859924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda747c7-b531-4252-987c-46b5fa50a8fc None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:34.893429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebe6fe-3916-4bed-8d26-47798aa9c11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:34.894033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebe6fe-3916-4bed-8d26-47798aa9c11c None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:34.894317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebe6fe-3916-4bed-8d26-47798aa9c11c None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:34.894622 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebe6fe-3916-4bed-8d26-47798aa9c11c None admin] PATH_INFO: `/v3/services` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:34.895258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebe6fe-3916-4bed-8d26-47798aa9c11c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:34.895565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebe6fe-3916-4bed-8d26-47798aa9c11c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:34.896938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebe6fe-3916-4bed-8d26-47798aa9c11c None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:34.901320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 635/1292] 10.222.0.82 () {56 vars in 1116 bytes} [Wed Aug 24 03:34:34 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:34:36.835718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8daf-19a7-4579-bca7-9d6002b6d79b None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:36.836248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8daf-19a7-4579-bca7-9d6002b6d79b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:36.836248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8daf-19a7-4579-bca7-9d6002b6d79b None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:36.837602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 658/1293] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:36.843196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68b7be-9ff5-417e-932d-758260c7c65f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:36.843633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68b7be-9ff5-417e-932d-758260c7c65f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:36.843756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68b7be-9ff5-417e-932d-758260c7c65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:36.861687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-cf68b7be-9ff5-417e-932d-758260c7c65f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:36.862900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cf68b7be-9ff5-417e-932d-758260c7c65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:36.893173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:36.931559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 636/1294] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:37.199091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8daf-19a7-4579-bca7-9d6002b6d79b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:37.231352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:37.302582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8daf-19a7-4579-bca7-9d6002b6d79b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:37.335528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144e45a-1966-47a7-a8a1-c0dd75edd8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:37.336153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144e45a-1966-47a7-a8a1-c0dd75edd8f8 None admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:37.336457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144e45a-1966-47a7-a8a1-c0dd75edd8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:37.336736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144e45a-1966-47a7-a8a1-c0dd75edd8f8 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:37.337377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144e45a-1966-47a7-a8a1-c0dd75edd8f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:37.338784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144e45a-1966-47a7-a8a1-c0dd75edd8f8 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:37.476154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 659/1295] 10.222.0.82 () {60 vars in 1141 bytes} [Wed Aug 24 03:34:37 2022] POST /identity/v3/services => generated 246 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:34:39.431660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d6a09-1020-42f8-b463-44643ec58eec None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:39.432190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d6a09-1020-42f8-b463-44643ec58eec None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:39.432190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d6a09-1020-42f8-b463-44643ec58eec None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:39.433742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 637/1296] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:39.438742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31234ce3-9c28-48cb-87ba-7592b5718ec1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:39.439153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31234ce3-9c28-48cb-87ba-7592b5718ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:39.439389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31234ce3-9c28-48cb-87ba-7592b5718ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:39.457774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-31234ce3-9c28-48cb-87ba-7592b5718ec1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:39.458981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-31234ce3-9c28-48cb-87ba-7592b5718ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:39.533312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 660/1297] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:39.800361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d6a09-1020-42f8-b463-44643ec58eec None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:39.905085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d6a09-1020-42f8-b463-44643ec58eec None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:39.939672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe7622-517a-4c09-805d-97be3aa6f4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:39.940304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe7622-517a-4c09-805d-97be3aa6f4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:39.940590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe7622-517a-4c09-805d-97be3aa6f4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:39.940868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe7622-517a-4c09-805d-97be3aa6f4b2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:39.941453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe7622-517a-4c09-805d-97be3aa6f4b2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:39.942951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe7622-517a-4c09-805d-97be3aa6f4b2 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-9cfe7622-517a-4c09-805d-97be3aa6f4b2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:39.951860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:39.952860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:39.953810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 638/1298] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:34:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:39.959838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31234ce3-9c28-48cb-87ba-7592b5718ec1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:39.978415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:40.061647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31234ce3-9c28-48cb-87ba-7592b5718ec1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:40.098223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff975cee-d3a5-42a0-8e21-3e03ed2fa203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:40.098997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff975cee-d3a5-42a0-8e21-3e03ed2fa203 None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:40.099278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff975cee-d3a5-42a0-8e21-3e03ed2fa203 None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:40.099582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff975cee-d3a5-42a0-8e21-3e03ed2fa203 None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:40.100223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff975cee-d3a5-42a0-8e21-3e03ed2fa203 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:40.100530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff975cee-d3a5-42a0-8e21-3e03ed2fa203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:40.101905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff975cee-d3a5-42a0-8e21-3e03ed2fa203 None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:40.106986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 661/1299] 10.222.0.82 () {56 vars in 1116 bytes} [Wed Aug 24 03:34:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:34:40.113368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe7622-517a-4c09-805d-97be3aa6f4b2 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:40.147332 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:40.238815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe7622-517a-4c09-805d-97be3aa6f4b2 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:40.274983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99c3f4-e362-4414-8355-dafb70b0df44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:40.275600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99c3f4-e362-4414-8355-dafb70b0df44 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:40.275884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99c3f4-e362-4414-8355-dafb70b0df44 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:40.276160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99c3f4-e362-4414-8355-dafb70b0df44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:40.276868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99c3f4-e362-4414-8355-dafb70b0df44 None admin] RBAC: Adding query filter params (interface=public, service_id=c9fbade09de84f39b16c6e018adf388c, region_id=RegionOne) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:40.277187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99c3f4-e362-4414-8355-dafb70b0df44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:40.278578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99c3f4-e362-4414-8355-dafb70b0df44 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:40.283998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 639/1300] 10.222.0.82 () {56 vars in 1251 bytes} [Wed Aug 24 03:34:40 2022] GET /identity/v3/endpoints?service_id=c9fbade09de84f39b16c6e018adf388c&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:40.289775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff975cee-d3a5-42a0-8e21-3e03ed2fa203 None admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:40.371579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:40.389916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff975cee-d3a5-42a0-8e21-3e03ed2fa203 None admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:40.425834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b949f0-d949-4c05-bd55-5559231fa2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:40.426434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b949f0-d949-4c05-bd55-5559231fa2fb None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:40.426741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b949f0-d949-4c05-bd55-5559231fa2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:40.427012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b949f0-d949-4c05-bd55-5559231fa2fb None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:40.427675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b949f0-d949-4c05-bd55-5559231fa2fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:40.429135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b949f0-d949-4c05-bd55-5559231fa2fb None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:40.435722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 662/1301] 10.222.0.82 () {56 vars in 1087 bytes} [Wed Aug 24 03:34:40 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:34:42.329120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb64576-9f6c-4cd8-8881-5cc93f48fa42 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:42.330238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb64576-9f6c-4cd8-8881-5cc93f48fa42 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:42.330238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb64576-9f6c-4cd8-8881-5cc93f48fa42 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:42.331761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 640/1302] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:42.337154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298fd50-4c7d-4d81-9141-422714809413 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:42.337599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298fd50-4c7d-4d81-9141-422714809413 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:42.337700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298fd50-4c7d-4d81-9141-422714809413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:42.356436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-8298fd50-4c7d-4d81-9141-422714809413 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:42.357638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8298fd50-4c7d-4d81-9141-422714809413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:42.428226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 663/1303] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:34:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:34:42.696382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb64576-9f6c-4cd8-8881-5cc93f48fa42 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:42.796707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:42.803955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb64576-9f6c-4cd8-8881-5cc93f48fa42 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:42.838515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eef8ab-0d0e-47ff-9263-53726dfc8703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:42.839169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eef8ab-0d0e-47ff-9263-53726dfc8703 None admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:42.839453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eef8ab-0d0e-47ff-9263-53726dfc8703 None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:42.839785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eef8ab-0d0e-47ff-9263-53726dfc8703 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:42.840372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eef8ab-0d0e-47ff-9263-53726dfc8703 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:42.841752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eef8ab-0d0e-47ff-9263-53726dfc8703 None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application [None req-04eef8ab-0d0e-47ff-9263-53726dfc8703 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:34:42.850485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:34:42.851772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application with Lock( Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: ERROR keystone.server.flask.application Aug 24 03:34:42.852746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 641/1304] 10.222.0.82 () {56 vars in 1117 bytes} [Wed Aug 24 03:34:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:34:42.857802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298fd50-4c7d-4d81-9141-422714809413 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:42.937394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:42.961464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298fd50-4c7d-4d81-9141-422714809413 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:42.995494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9d131-0a51-420c-b415-fa726d450dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:42.996096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9d131-0a51-420c-b415-fa726d450dbd None admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:42.996373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9d131-0a51-420c-b415-fa726d450dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:42.996646 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9d131-0a51-420c-b415-fa726d450dbd None admin] PATH_INFO: `/v3/services` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:42.997269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9d131-0a51-420c-b415-fa726d450dbd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:34:42.997570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9d131-0a51-420c-b415-fa726d450dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:42.998971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9d131-0a51-420c-b415-fa726d450dbd None admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:43.005173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 664/1305] 10.222.0.82 () {56 vars in 1116 bytes} [Wed Aug 24 03:34:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:43.012037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eef8ab-0d0e-47ff-9263-53726dfc8703 None admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:43.113476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eef8ab-0d0e-47ff-9263-53726dfc8703 None admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:43.146475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5c33d-a908-41ef-b1e8-7ea5eb56648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:43.147104 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c33d-a908-41ef-b1e8-7ea5eb56648a None admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:43.147388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c33d-a908-41ef-b1e8-7ea5eb56648a None admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:43.147661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c33d-a908-41ef-b1e8-7ea5eb56648a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:43.148302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5c33d-a908-41ef-b1e8-7ea5eb56648a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:43.149675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5c33d-a908-41ef-b1e8-7ea5eb56648a None admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:43.247430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 642/1306] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:34:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:34:53.029389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=30 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:53.242579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:35.759880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ad345-7d76-45f3-977f-0ce09ff4b580 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:35.760346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ad345-7d76-45f3-977f-0ce09ff4b580 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:35.760537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ad345-7d76-45f3-977f-0ce09ff4b580 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:35.838067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 665/1307] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:43:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:35.838816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d1d60-7f8a-46f8-83f0-dccd88ce5f22 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:35.838816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d1d60-7f8a-46f8-83f0-dccd88ce5f22 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:35.838816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d1d60-7f8a-46f8-83f0-dccd88ce5f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:35.850432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e85d1d60-7f8a-46f8-83f0-dccd88ce5f22 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:35.850432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e85d1d60-7f8a-46f8-83f0-dccd88ce5f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:36.030603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 643/1308] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:36.319255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced0ace-00ed-4c8b-91b4-22d4dad3b297 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:36.319544 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced0ace-00ed-4c8b-91b4-22d4dad3b297 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:36.319976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced0ace-00ed-4c8b-91b4-22d4dad3b297 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:36.321550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 666/1309] 10.222.0.82 () {54 vars in 965 bytes} [Wed Aug 24 03:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:36.331935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6285f3-7df4-41b3-88c7-c7bcac17b050 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:36.333037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6285f3-7df4-41b3-88c7-c7bcac17b050 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:36.333725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6285f3-7df4-41b3-88c7-c7bcac17b050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:36.420403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-9b6285f3-7df4-41b3-88c7-c7bcac17b050 None None] MFA Rules not processed for user `f81cc77e85f84d358471dd9efab45cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:36.422053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9b6285f3-7df4-41b3-88c7-c7bcac17b050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:36.445965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:36.508669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 644/1310] 10.222.0.82 () {58 vars in 1062 bytes} [Wed Aug 24 03:43:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:36.518893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced0ace-00ed-4c8b-91b4-22d4dad3b297 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:36.520653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-dced0ace-00ed-4c8b-91b4-22d4dad3b297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:36.652374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced0ace-00ed-4c8b-91b4-22d4dad3b297 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:36.736114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c517e8-f2e2-4674-a291-59c42736c706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:36.737084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c517e8-f2e2-4674-a291-59c42736c706 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:36.737696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c517e8-f2e2-4674-a291-59c42736c706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:36.738169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c517e8-f2e2-4674-a291-59c42736c706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:36.740140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-99c517e8-f2e2-4674-a291-59c42736c706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:36.801209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:36.863242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c517e8-f2e2-4674-a291-59c42736c706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:36.864996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c517e8-f2e2-4674-a291-59c42736c706 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:36.889220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 667/1311] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:39.146766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ec7ff-7d3b-4966-9fe8-52a8cd51e258 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:39.147333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ec7ff-7d3b-4966-9fe8-52a8cd51e258 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:39.147333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ec7ff-7d3b-4966-9fe8-52a8cd51e258 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:39.148819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 645/1312] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:39.154857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea15f53-0a3c-4201-b278-a0e46b400264 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:39.155254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea15f53-0a3c-4201-b278-a0e46b400264 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:39.155355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea15f53-0a3c-4201-b278-a0e46b400264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:39.248564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5ea15f53-0a3c-4201-b278-a0e46b400264 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:39.250012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5ea15f53-0a3c-4201-b278-a0e46b400264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:39.360536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 668/1313] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:39.642782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ec7ff-7d3b-4966-9fe8-52a8cd51e258 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:39.644409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b85ec7ff-7d3b-4966-9fe8-52a8cd51e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:39.751696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ec7ff-7d3b-4966-9fe8-52a8cd51e258 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:39.825367 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:39.828053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9f70-bc94-46a5-913f-5193a94c2cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:39.828667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9f70-bc94-46a5-913f-5193a94c2cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:39.828953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9f70-bc94-46a5-913f-5193a94c2cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:39.829229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9f70-bc94-46a5-913f-5193a94c2cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:39.830962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-339b9f70-bc94-46a5-913f-5193a94c2cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:39.943121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9f70-bc94-46a5-913f-5193a94c2cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:39.944592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9f70-bc94-46a5-913f-5193a94c2cfb service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:39.963964 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 646/1314] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:42.472805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7cab0-9e84-41cb-bafe-2f1aca96ec75 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:42.473361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7cab0-9e84-41cb-bafe-2f1aca96ec75 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:42.473361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7cab0-9e84-41cb-bafe-2f1aca96ec75 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:42.474732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 669/1315] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:42.480136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84693e92-34bf-49b8-b984-92da7e28e7e8 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:42.480578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84693e92-34bf-49b8-b984-92da7e28e7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:42.480975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84693e92-34bf-49b8-b984-92da7e28e7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:42.548982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-84693e92-34bf-49b8-b984-92da7e28e7e8 None None] MFA Rules not processed for user `3eb5ec627dc1453ca7e211a5b47289af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:42.550211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-84693e92-34bf-49b8-b984-92da7e28e7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:42.601150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:42.645844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 647/1316] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:43:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:42.931604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7cab0-9e84-41cb-bafe-2f1aca96ec75 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:42.933178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-59e7cab0-9e84-41cb-bafe-2f1aca96ec75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:42.943746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:43.039069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7cab0-9e84-41cb-bafe-2f1aca96ec75 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:43.111726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb061936-4676-4736-bc63-a9162707e61d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:43.112335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb061936-4676-4736-bc63-a9162707e61d service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:43.112619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb061936-4676-4736-bc63-a9162707e61d service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:43.112898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb061936-4676-4736-bc63-a9162707e61d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:43.114538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-eb061936-4676-4736-bc63-a9162707e61d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:43.176726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:43.223596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb061936-4676-4736-bc63-a9162707e61d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:43.225048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb061936-4676-4736-bc63-a9162707e61d service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:43.243211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 670/1317] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:46.054946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089b4bd-c937-4051-863c-95b0421d2ddf None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:46.055435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089b4bd-c937-4051-863c-95b0421d2ddf None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:46.055529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089b4bd-c937-4051-863c-95b0421d2ddf None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:46.056849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 648/1318] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:46.062837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281f67-5075-4668-9efc-6f528852b039 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:46.063292 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281f67-5075-4668-9efc-6f528852b039 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:46.063390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281f67-5075-4668-9efc-6f528852b039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:46.129136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-19281f67-5075-4668-9efc-6f528852b039 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:46.130357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-19281f67-5075-4668-9efc-6f528852b039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:46.226866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 671/1319] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:43:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:46.538523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e259f39-354b-42da-862b-da3372651f01 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:46.538813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e259f39-354b-42da-862b-da3372651f01 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:46.539096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e259f39-354b-42da-862b-da3372651f01 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:46.540435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 649/1320] 10.222.0.82 () {54 vars in 965 bytes} [Wed Aug 24 03:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:46.547178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8636b-661f-48a8-890e-be34e8e20943 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:46.547471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8636b-661f-48a8-890e-be34e8e20943 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:46.547753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8636b-661f-48a8-890e-be34e8e20943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:46.570230 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:46.623751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-42b8636b-661f-48a8-890e-be34e8e20943 None None] MFA Rules not processed for user `f81cc77e85f84d358471dd9efab45cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:46.624982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-42b8636b-661f-48a8-890e-be34e8e20943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:46.700185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 672/1321] 10.222.0.82 () {58 vars in 1062 bytes} [Wed Aug 24 03:43:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:46.709364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e259f39-354b-42da-862b-da3372651f01 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:46.710957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7e259f39-354b-42da-862b-da3372651f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:46.818122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e259f39-354b-42da-862b-da3372651f01 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:46.891040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc0d07-ecb0-4b03-a499-2c9f1d95403f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:46.891641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc0d07-ecb0-4b03-a499-2c9f1d95403f service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:46.891918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc0d07-ecb0-4b03-a499-2c9f1d95403f service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:46.892187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc0d07-ecb0-4b03-a499-2c9f1d95403f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:46.893773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-70cc0d07-ecb0-4b03-a499-2c9f1d95403f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:46.901044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:47.005484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc0d07-ecb0-4b03-a499-2c9f1d95403f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:47.006972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc0d07-ecb0-4b03-a499-2c9f1d95403f service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:47.025052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 650/1322] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:43:50.470458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13137800-393e-4427-821e-3c917530f373 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:50.471012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13137800-393e-4427-821e-3c917530f373 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:50.471134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13137800-393e-4427-821e-3c917530f373 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:50.472457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 673/1323] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:50.478076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e64a80-67bc-4f3b-917d-f7904bd088eb None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:50.478489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e64a80-67bc-4f3b-917d-f7904bd088eb None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:50.478792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e64a80-67bc-4f3b-917d-f7904bd088eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:50.546475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-89e64a80-67bc-4f3b-917d-f7904bd088eb None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:50.547712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-89e64a80-67bc-4f3b-917d-f7904bd088eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:50.605257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:50.644919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 651/1324] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:50.929261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13137800-393e-4427-821e-3c917530f373 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:50.930843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-13137800-393e-4427-821e-3c917530f373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:51.057005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13137800-393e-4427-821e-3c917530f373 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:51.070034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:51.139948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77fb84-e089-431c-b417-6fc447de32e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:51.140597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77fb84-e089-431c-b417-6fc447de32e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:51.140879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77fb84-e089-431c-b417-6fc447de32e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:51.141149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77fb84-e089-431c-b417-6fc447de32e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:51.142865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fa77fb84-e089-431c-b417-6fc447de32e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:51.264554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77fb84-e089-431c-b417-6fc447de32e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:51.266020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77fb84-e089-431c-b417-6fc447de32e0 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:51.285745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 674/1325] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:53.291002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4807ae9-150b-4706-b0b4-6e175e6aef3d None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:53.291595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4807ae9-150b-4706-b0b4-6e175e6aef3d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:53.291595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4807ae9-150b-4706-b0b4-6e175e6aef3d None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:53.293490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 652/1326] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:53.299753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518ccfb-c743-442f-aafd-5db87ec3921e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:53.300292 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518ccfb-c743-442f-aafd-5db87ec3921e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:53.300292 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518ccfb-c743-442f-aafd-5db87ec3921e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:53.357036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:53.376040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-1518ccfb-c743-442f-aafd-5db87ec3921e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:53.377276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1518ccfb-c743-442f-aafd-5db87ec3921e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:53.479645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 675/1327] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:53.765893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4807ae9-150b-4706-b0b4-6e175e6aef3d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:53.767500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c4807ae9-150b-4706-b0b4-6e175e6aef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:53.873497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4807ae9-150b-4706-b0b4-6e175e6aef3d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:53.948386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f07c7-411c-4b34-aa84-e788af792b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:53.949002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f07c7-411c-4b34-aa84-e788af792b0f service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:53.949942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f07c7-411c-4b34-aa84-e788af792b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:53.950445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f07c7-411c-4b34-aa84-e788af792b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:53.952162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-263f07c7-411c-4b34-aa84-e788af792b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:53.969709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:54.069403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f07c7-411c-4b34-aa84-e788af792b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:54.070923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f07c7-411c-4b34-aa84-e788af792b0f service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:54.091583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 653/1328] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:56.175150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e404011-a7a7-4f12-870e-52f6e063cc3e None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:56.175749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e404011-a7a7-4f12-870e-52f6e063cc3e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:56.175749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e404011-a7a7-4f12-870e-52f6e063cc3e None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:56.177212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 676/1329] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:56.182426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5671cfd-dcea-4519-be9e-4ddd82fe0b6a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:56.182878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5671cfd-dcea-4519-be9e-4ddd82fe0b6a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:56.183082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5671cfd-dcea-4519-be9e-4ddd82fe0b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:56.250975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f5671cfd-dcea-4519-be9e-4ddd82fe0b6a None None] MFA Rules not processed for user `3eb5ec627dc1453ca7e211a5b47289af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:56.252203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f5671cfd-dcea-4519-be9e-4ddd82fe0b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:56.319427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:56.357374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 654/1330] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:43:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:56.642457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e404011-a7a7-4f12-870e-52f6e063cc3e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:56.644052 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4e404011-a7a7-4f12-870e-52f6e063cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:56.747713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e404011-a7a7-4f12-870e-52f6e063cc3e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:56.768010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:56.823092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71717e1b-fb3e-4b73-93ad-89da4fffb5ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:56.823714 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71717e1b-fb3e-4b73-93ad-89da4fffb5ee service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:56.823999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71717e1b-fb3e-4b73-93ad-89da4fffb5ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:56.824895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71717e1b-fb3e-4b73-93ad-89da4fffb5ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:56.826645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-71717e1b-fb3e-4b73-93ad-89da4fffb5ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:56.943050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71717e1b-fb3e-4b73-93ad-89da4fffb5ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:56.944507 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71717e1b-fb3e-4b73-93ad-89da4fffb5ee service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:56.963507 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 677/1331] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:59.482940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b89cb7-c0fe-4f18-a09a-77b63d225a04 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:59.483533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b89cb7-c0fe-4f18-a09a-77b63d225a04 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:59.483533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b89cb7-c0fe-4f18-a09a-77b63d225a04 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:59.484854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 655/1332] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:43:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:43:59.490392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9cdef-4d49-42b6-ba66-8db2a0cdf623 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:59.490856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9cdef-4d49-42b6-ba66-8db2a0cdf623 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:59.491003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9cdef-4d49-42b6-ba66-8db2a0cdf623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:59.560505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:59.563198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-28f9cdef-4d49-42b6-ba66-8db2a0cdf623 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:59.564425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-28f9cdef-4d49-42b6-ba66-8db2a0cdf623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:59.665007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 678/1333] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:59.946561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b89cb7-c0fe-4f18-a09a-77b63d225a04 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:59.948111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-70b89cb7-c0fe-4f18-a09a-77b63d225a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:00.057194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b89cb7-c0fe-4f18-a09a-77b63d225a04 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:00.140090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57efb58-784d-4c71-822f-470900c0ce9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:00.140703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57efb58-784d-4c71-822f-470900c0ce9b service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:00.140981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57efb58-784d-4c71-822f-470900c0ce9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:00.141248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57efb58-784d-4c71-822f-470900c0ce9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:00.142901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c57efb58-784d-4c71-822f-470900c0ce9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:00.172440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:00.258005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57efb58-784d-4c71-822f-470900c0ce9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:00.259412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57efb58-784d-4c71-822f-470900c0ce9b service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:00.278825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 656/1334] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:02.243905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4adbc-4d7e-4874-b597-6c7e7c271c52 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:02.244469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4adbc-4d7e-4874-b597-6c7e7c271c52 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:02.244469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4adbc-4d7e-4874-b597-6c7e7c271c52 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:02.245801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 679/1335] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:02.251123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddec84-31a4-4780-8d55-8f8bb8ad5d7e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:02.251569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddec84-31a4-4780-8d55-8f8bb8ad5d7e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:02.251685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddec84-31a4-4780-8d55-8f8bb8ad5d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:02.319911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-56ddec84-31a4-4780-8d55-8f8bb8ad5d7e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:02.321133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-56ddec84-31a4-4780-8d55-8f8bb8ad5d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:02.396750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:02.419497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 657/1336] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:02.705772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4adbc-4d7e-4874-b597-6c7e7c271c52 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:02.707368 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-73a4adbc-4d7e-4874-b597-6c7e7c271c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:02.814265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4adbc-4d7e-4874-b597-6c7e7c271c52 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:02.842528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:02.888719 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581ac65-88ca-419b-8fc3-ee0e3b11b27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:02.889331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581ac65-88ca-419b-8fc3-ee0e3b11b27a service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:02.889614 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581ac65-88ca-419b-8fc3-ee0e3b11b27a service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:02.889890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581ac65-88ca-419b-8fc3-ee0e3b11b27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:02.891518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c581ac65-88ca-419b-8fc3-ee0e3b11b27a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:02.998106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581ac65-88ca-419b-8fc3-ee0e3b11b27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:02.999519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581ac65-88ca-419b-8fc3-ee0e3b11b27a service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:03.019213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 680/1337] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:05.715686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726e52b-81a0-4d76-8332-c28a6550aa87 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:05.716257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726e52b-81a0-4d76-8332-c28a6550aa87 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:05.716257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726e52b-81a0-4d76-8332-c28a6550aa87 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:05.717585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 658/1338] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:05.722878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6680b-f72d-468b-a04d-119cd86fa391 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:05.723279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6680b-f72d-468b-a04d-119cd86fa391 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:05.723432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6680b-f72d-468b-a04d-119cd86fa391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:05.790087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a7e6680b-f72d-468b-a04d-119cd86fa391 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:05.791339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a7e6680b-f72d-468b-a04d-119cd86fa391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:05.800079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:05.889740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 681/1339] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:06.175448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726e52b-81a0-4d76-8332-c28a6550aa87 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:06.177032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9726e52b-81a0-4d76-8332-c28a6550aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:06.280314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726e52b-81a0-4d76-8332-c28a6550aa87 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:06.353319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e0605-e945-436f-90c8-88256adb1067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:06.353959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e0605-e945-436f-90c8-88256adb1067 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:06.354239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e0605-e945-436f-90c8-88256adb1067 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:06.354537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e0605-e945-436f-90c8-88256adb1067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:06.356912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-439e0605-e945-436f-90c8-88256adb1067 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:06.391842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:06.467408 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e0605-e945-436f-90c8-88256adb1067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:06.468875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e0605-e945-436f-90c8-88256adb1067 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:06.488563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 659/1340] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:10.729162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4d219-f086-4a16-a603-7611632898b0 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:10.729737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4d219-f086-4a16-a603-7611632898b0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:10.729737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4d219-f086-4a16-a603-7611632898b0 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:10.731222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 682/1341] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:10.736740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb815f-5133-46a8-93ca-46cb2d93b3f7 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:10.737190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb815f-5133-46a8-93ca-46cb2d93b3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:10.737301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb815f-5133-46a8-93ca-46cb2d93b3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:10.809654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-0dcb815f-5133-46a8-93ca-46cb2d93b3f7 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:10.810927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0dcb815f-5133-46a8-93ca-46cb2d93b3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:10.905558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:10.913483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 660/1342] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:11.205045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4d219-f086-4a16-a603-7611632898b0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:11.206644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-aef4d219-f086-4a16-a603-7611632898b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:11.328142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4d219-f086-4a16-a603-7611632898b0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:11.367819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:11.406845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ee728-67b8-417d-b259-030c4d8d123c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:11.407491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ee728-67b8-417d-b259-030c4d8d123c service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:11.407767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ee728-67b8-417d-b259-030c4d8d123c service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:11.408039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ee728-67b8-417d-b259-030c4d8d123c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:11.409631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-997ee728-67b8-417d-b259-030c4d8d123c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:11.522609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ee728-67b8-417d-b259-030c4d8d123c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:11.524004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ee728-67b8-417d-b259-030c4d8d123c service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:11.544621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 683/1343] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:14.757110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3019063-02f6-465c-9198-519dd799198f None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:14.757651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3019063-02f6-465c-9198-519dd799198f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:14.757651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3019063-02f6-465c-9198-519dd799198f None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:14.759055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 661/1344] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:14.765088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47491b-4f0a-456c-8a00-12f8fe6c6b51 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:14.765582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47491b-4f0a-456c-8a00-12f8fe6c6b51 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:14.765678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47491b-4f0a-456c-8a00-12f8fe6c6b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:14.837229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-fa47491b-4f0a-456c-8a00-12f8fe6c6b51 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:14.838436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fa47491b-4f0a-456c-8a00-12f8fe6c6b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:14.857507 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:14.946680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 684/1345] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:15.238581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3019063-02f6-465c-9198-519dd799198f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:15.240137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c3019063-02f6-465c-9198-519dd799198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:15.357319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3019063-02f6-465c-9198-519dd799198f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:15.445743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae30bbc-9a5d-4fe4-a345-426a7d0713d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:15.446355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30bbc-9a5d-4fe4-a345-426a7d0713d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:15.446792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30bbc-9a5d-4fe4-a345-426a7d0713d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:15.447080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30bbc-9a5d-4fe4-a345-426a7d0713d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:15.448698 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-eae30bbc-9a5d-4fe4-a345-426a7d0713d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:15.495030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:15.572608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae30bbc-9a5d-4fe4-a345-426a7d0713d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:15.574154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae30bbc-9a5d-4fe4-a345-426a7d0713d3 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:15.595173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 662/1346] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:20.057109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45647911-ccdd-44e5-8ba2-3df72c8ef2bc None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:20.057654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45647911-ccdd-44e5-8ba2-3df72c8ef2bc None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:20.057654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45647911-ccdd-44e5-8ba2-3df72c8ef2bc None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:20.059102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 685/1347] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:20.065720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf0d06-4d69-47e2-b5f1-ad99bd15f3aa None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:20.066174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf0d06-4d69-47e2-b5f1-ad99bd15f3aa None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:20.066361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf0d06-4d69-47e2-b5f1-ad99bd15f3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:20.147156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7dbf0d06-4d69-47e2-b5f1-ad99bd15f3aa None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:20.148418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7dbf0d06-4d69-47e2-b5f1-ad99bd15f3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:20.262882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:20.265501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 663/1348] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:20.597564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45647911-ccdd-44e5-8ba2-3df72c8ef2bc None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:20.599196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-45647911-ccdd-44e5-8ba2-3df72c8ef2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:20.719383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45647911-ccdd-44e5-8ba2-3df72c8ef2bc None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:20.772672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:20.801169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9befa-4a55-4cd3-8a80-84c7c66b6d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:20.802414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9befa-4a55-4cd3-8a80-84c7c66b6d02 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:20.802414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9befa-4a55-4cd3-8a80-84c7c66b6d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:20.802414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9befa-4a55-4cd3-8a80-84c7c66b6d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:20.803955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-88b9befa-4a55-4cd3-8a80-84c7c66b6d02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:20.945775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9befa-4a55-4cd3-8a80-84c7c66b6d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:20.947271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9befa-4a55-4cd3-8a80-84c7c66b6d02 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:20.968133 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 686/1349] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:26.117670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03699c3-3dfc-4944-a1ea-b549900ee199 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:26.117670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03699c3-3dfc-4944-a1ea-b549900ee199 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:26.118211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03699c3-3dfc-4944-a1ea-b549900ee199 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:26.119484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 664/1350] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:26.125376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff2dff-d605-41da-9dd1-b44ce8133c3b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:26.125902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff2dff-d605-41da-9dd1-b44ce8133c3b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:26.125994 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff2dff-d605-41da-9dd1-b44ce8133c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:26.218153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5dff2dff-d605-41da-9dd1-b44ce8133c3b None None] MFA Rules not processed for user `cff23ae056ff452588cd0b6520e96962`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:26.219406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5dff2dff-d605-41da-9dd1-b44ce8133c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:26.240366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:26.414442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 687/1351] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:44:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:30.659634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263e945-f200-48ee-ac9d-2f9431898310 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:30.660505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263e945-f200-48ee-ac9d-2f9431898310 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:30.660787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263e945-f200-48ee-ac9d-2f9431898310 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:30.662130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 665/1352] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:44:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:30.667252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0279d9-8af2-414d-a0e9-6b2eec0cbd72 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:30.667697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0279d9-8af2-414d-a0e9-6b2eec0cbd72 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:30.667807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0279d9-8af2-414d-a0e9-6b2eec0cbd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:30.755073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-fb0279d9-8af2-414d-a0e9-6b2eec0cbd72 None None] MFA Rules not processed for user `cff23ae056ff452588cd0b6520e96962`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:30.756594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fb0279d9-8af2-414d-a0e9-6b2eec0cbd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:30.841417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:30.880764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 688/1353] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:44:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:32.288482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa671e-d6ef-49ac-b50f-a3379b99598e None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:32.289017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa671e-d6ef-49ac-b50f-a3379b99598e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:32.289017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa671e-d6ef-49ac-b50f-a3379b99598e None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:32.290302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 666/1354] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:32.295529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadbafc-03d9-4295-a42c-f939b1cf4265 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:32.296008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadbafc-03d9-4295-a42c-f939b1cf4265 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:32.296197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadbafc-03d9-4295-a42c-f939b1cf4265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:32.371634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-fdadbafc-03d9-4295-a42c-f939b1cf4265 None None] MFA Rules not processed for user `cff23ae056ff452588cd0b6520e96962`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:32.372851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fdadbafc-03d9-4295-a42c-f939b1cf4265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:32.483234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 689/1355] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:44:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:35.976026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b7868-24c0-40e1-906c-736d90afdf20 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:35.976613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b7868-24c0-40e1-906c-736d90afdf20 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:35.976713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b7868-24c0-40e1-906c-736d90afdf20 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:35.978091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 667/1356] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:35.984195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd47864-6407-423f-b435-6b099f9b1084 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:35.984651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd47864-6407-423f-b435-6b099f9b1084 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:35.984875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd47864-6407-423f-b435-6b099f9b1084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:36.026016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:36.066414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-dfd47864-6407-423f-b435-6b099f9b1084 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:36.069183 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-dfd47864-6407-423f-b435-6b099f9b1084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:36.209156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 690/1357] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:44:36.546846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8313a5d-4f65-4c9e-ba30-dd5112699f08 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:36.547123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8313a5d-4f65-4c9e-ba30-dd5112699f08 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:36.547642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8313a5d-4f65-4c9e-ba30-dd5112699f08 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:36.549146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 668/1358] 10.222.0.82 () {54 vars in 962 bytes} [Wed Aug 24 03:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:36.555966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a6f77-07f7-4eba-9142-ca8890016ecd None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:36.556468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a6f77-07f7-4eba-9142-ca8890016ecd None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:36.556886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a6f77-07f7-4eba-9142-ca8890016ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:36.649018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-bf2a6f77-07f7-4eba-9142-ca8890016ecd None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:36.650936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bf2a6f77-07f7-4eba-9142-ca8890016ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:36.709054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:36.781234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 691/1359] 10.222.0.82 () {58 vars in 1059 bytes} [Wed Aug 24 03:44:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:36.789596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8313a5d-4f65-4c9e-ba30-dd5112699f08 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:36.791209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d8313a5d-4f65-4c9e-ba30-dd5112699f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:36.953180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8313a5d-4f65-4c9e-ba30-dd5112699f08 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:37.050013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:37.099364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1399be-a0a7-4928-84d0-d75dfc0741b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:37.100317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1399be-a0a7-4928-84d0-d75dfc0741b0 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:37.100604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1399be-a0a7-4928-84d0-d75dfc0741b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:37.100879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1399be-a0a7-4928-84d0-d75dfc0741b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:37.102481 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bb1399be-a0a7-4928-84d0-d75dfc0741b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:37.196133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1399be-a0a7-4928-84d0-d75dfc0741b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:37.197743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1399be-a0a7-4928-84d0-d75dfc0741b0 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:37.217422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 669/1360] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:44:39.418238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cba06-57bb-4249-8859-2a91d0b2a655 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:39.419487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cba06-57bb-4249-8859-2a91d0b2a655 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:39.420087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cba06-57bb-4249-8859-2a91d0b2a655 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:39.427809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 692/1361] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:39.428398 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2274-5646-4408-9373-ca45d7a8f560 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:39.429200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2274-5646-4408-9373-ca45d7a8f560 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:39.429200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2274-5646-4408-9373-ca45d7a8f560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:39.507131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-827c2274-5646-4408-9373-ca45d7a8f560 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:39.508350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-827c2274-5646-4408-9373-ca45d7a8f560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:39.549668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:39.614625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 670/1362] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:44:39.903516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cba06-57bb-4249-8859-2a91d0b2a655 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:39.905217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ae4cba06-57bb-4249-8859-2a91d0b2a655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:40.091481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cba06-57bb-4249-8859-2a91d0b2a655 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:40.104351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:40.225998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31350bee-4908-4f2d-aa0b-4b862f808f12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:40.226642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31350bee-4908-4f2d-aa0b-4b862f808f12 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:40.226928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31350bee-4908-4f2d-aa0b-4b862f808f12 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:40.227205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31350bee-4908-4f2d-aa0b-4b862f808f12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:40.228827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-31350bee-4908-4f2d-aa0b-4b862f808f12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:40.331295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31350bee-4908-4f2d-aa0b-4b862f808f12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:40.332925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31350bee-4908-4f2d-aa0b-4b862f808f12 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:40.355550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 693/1363] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:42.845418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8badc-daf6-4a94-89f7-f453090dbd50 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:42.845946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8badc-daf6-4a94-89f7-f453090dbd50 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:42.846060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8badc-daf6-4a94-89f7-f453090dbd50 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:42.847361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 671/1364] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:42.853147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3745b6-539e-4fff-a213-608d415ad1ff None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:42.853581 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3745b6-539e-4fff-a213-608d415ad1ff None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:42.853765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3745b6-539e-4fff-a213-608d415ad1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:42.871301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:42.928622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-2d3745b6-539e-4fff-a213-608d415ad1ff None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:42.929842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2d3745b6-539e-4fff-a213-608d415ad1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:43.046558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 694/1365] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:43.345035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f063705-051c-488b-8047-77c3fc8e8b7f None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:43.345588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f063705-051c-488b-8047-77c3fc8e8b7f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:43.346107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f063705-051c-488b-8047-77c3fc8e8b7f None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:43.347732 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 672/1366] 10.222.0.82 () {54 vars in 962 bytes} [Wed Aug 24 03:44:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:43.353582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2804a4e-eecb-49af-8eb9-d5b13a7b0320 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:43.353933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2804a4e-eecb-49af-8eb9-d5b13a7b0320 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:43.354216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2804a4e-eecb-49af-8eb9-d5b13a7b0320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:43.437349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a2804a4e-eecb-49af-8eb9-d5b13a7b0320 None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:43.438574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a2804a4e-eecb-49af-8eb9-d5b13a7b0320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:43.475204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:43.562222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 695/1367] 10.222.0.82 () {58 vars in 1059 bytes} [Wed Aug 24 03:44:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:43.570261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f063705-051c-488b-8047-77c3fc8e8b7f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:43.571820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4f063705-051c-488b-8047-77c3fc8e8b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:43.739707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f063705-051c-488b-8047-77c3fc8e8b7f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:43.761750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:43.861193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a769f5-1561-42f9-af18-0d58f77ec090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:43.861860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a769f5-1561-42f9-af18-0d58f77ec090 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:43.862094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a769f5-1561-42f9-af18-0d58f77ec090 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:43.862352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a769f5-1561-42f9-af18-0d58f77ec090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:43.863988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a6a769f5-1561-42f9-af18-0d58f77ec090 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:43.964654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a769f5-1561-42f9-af18-0d58f77ec090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:43.966539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a769f5-1561-42f9-af18-0d58f77ec090 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:43.986881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 673/1368] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:44.507176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee8e2f-657e-457c-82e8-ccd8aa96f0e1 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:44.507722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee8e2f-657e-457c-82e8-ccd8aa96f0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:44.507722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee8e2f-657e-457c-82e8-ccd8aa96f0e1 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:44.509135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 696/1369] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:44.514461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57116e-4189-4195-9345-fb399c5a8b13 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:44.514774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57116e-4189-4195-9345-fb399c5a8b13 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:44.515054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57116e-4189-4195-9345-fb399c5a8b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:44.549429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:44.587689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-1a57116e-4189-4195-9345-fb399c5a8b13 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:44.588906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1a57116e-4189-4195-9345-fb399c5a8b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:44.696395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 674/1370] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:45.277631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b834f92-e71d-4584-a1ed-08c4000f2111 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:45.278821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b834f92-e71d-4584-a1ed-08c4000f2111 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:45.279411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b834f92-e71d-4584-a1ed-08c4000f2111 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:45.281022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 697/1371] 10.222.0.82 () {54 vars in 964 bytes} [Wed Aug 24 03:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:45.286258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcaba5-5116-41e2-883d-4269f1068f56 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:45.286847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcaba5-5116-41e2-883d-4269f1068f56 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:45.286847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcaba5-5116-41e2-883d-4269f1068f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:45.372292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-24dcaba5-5116-41e2-883d-4269f1068f56 None None] MFA Rules not processed for user `c89caed732274718aeaa0d8ffeef571b`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:45.373814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-24dcaba5-5116-41e2-883d-4269f1068f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:45.425259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:45.455852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 675/1372] 10.222.0.82 () {58 vars in 1061 bytes} [Wed Aug 24 03:44:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:45.463364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b834f92-e71d-4584-a1ed-08c4000f2111 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:45.465007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8b834f92-e71d-4584-a1ed-08c4000f2111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:45.577720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b834f92-e71d-4584-a1ed-08c4000f2111 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:45.620018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:45.657187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50cd03-acb5-48cc-b235-a61ad809ff45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:45.658649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50cd03-acb5-48cc-b235-a61ad809ff45 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:45.659086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50cd03-acb5-48cc-b235-a61ad809ff45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:45.659537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50cd03-acb5-48cc-b235-a61ad809ff45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:45.663527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-dc50cd03-acb5-48cc-b235-a61ad809ff45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:45.783300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50cd03-acb5-48cc-b235-a61ad809ff45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:45.784776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50cd03-acb5-48cc-b235-a61ad809ff45 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:45.814807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 698/1373] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:46.166566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a183049-8bde-4115-ba05-939e719a4186 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:46.166566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a183049-8bde-4115-ba05-939e719a4186 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:46.166566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a183049-8bde-4115-ba05-939e719a4186 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:46.171017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 676/1374] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:46.173531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23281234-99c8-4e35-bdc0-5cd761acd0c5 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:46.173824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23281234-99c8-4e35-bdc0-5cd761acd0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:46.174102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23281234-99c8-4e35-bdc0-5cd761acd0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:46.266740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-23281234-99c8-4e35-bdc0-5cd761acd0c5 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:46.267967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-23281234-99c8-4e35-bdc0-5cd761acd0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:46.288751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:46.366104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 699/1375] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:46.669108 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a183049-8bde-4115-ba05-939e719a4186 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:46.670669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1a183049-8bde-4115-ba05-939e719a4186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:46.823045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a183049-8bde-4115-ba05-939e719a4186 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:46.877772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:46.942364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13609a-cdf4-48d7-8167-8fadefd91d03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:46.942915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13609a-cdf4-48d7-8167-8fadefd91d03 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:46.943197 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13609a-cdf4-48d7-8167-8fadefd91d03 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:46.943473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13609a-cdf4-48d7-8167-8fadefd91d03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:46.945089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fd13609a-cdf4-48d7-8167-8fadefd91d03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:47.038139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13609a-cdf4-48d7-8167-8fadefd91d03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:47.039801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13609a-cdf4-48d7-8167-8fadefd91d03 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:47.058850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 677/1376] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:49.255425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaa3ce-c9c9-4eeb-8613-69c34015f361 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:49.255972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaa3ce-c9c9-4eeb-8613-69c34015f361 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:49.255972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaa3ce-c9c9-4eeb-8613-69c34015f361 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:49.257350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 700/1377] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:49.263189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9e09c-9559-451d-926e-11d504c9c0b5 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:49.263704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9e09c-9559-451d-926e-11d504c9c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:49.263704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9e09c-9559-451d-926e-11d504c9c0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:49.335199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d8a9e09c-9559-451d-926e-11d504c9c0b5 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:49.336509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d8a9e09c-9559-451d-926e-11d504c9c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:49.343986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:49.452968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 678/1378] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:49.762798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaa3ce-c9c9-4eeb-8613-69c34015f361 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:49.765017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-75eaa3ce-c9c9-4eeb-8613-69c34015f361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:49.927288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaa3ce-c9c9-4eeb-8613-69c34015f361 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:49.931484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:50.061202 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fdb49-08be-4c08-b10e-a2cef921a867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:50.061819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fdb49-08be-4c08-b10e-a2cef921a867 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:50.062100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fdb49-08be-4c08-b10e-a2cef921a867 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:50.062373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fdb49-08be-4c08-b10e-a2cef921a867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:50.064036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-125fdb49-08be-4c08-b10e-a2cef921a867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:50.161533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fdb49-08be-4c08-b10e-a2cef921a867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:50.163250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fdb49-08be-4c08-b10e-a2cef921a867 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:50.182323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 701/1379] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:52.359316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd82561-0280-4800-bd81-184260c42ad4 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:52.359901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd82561-0280-4800-bd81-184260c42ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:52.359901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd82561-0280-4800-bd81-184260c42ad4 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:52.361898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 679/1380] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:52.368444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79983d36-6b2e-4550-8d26-5b431e7997be None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:52.368967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79983d36-6b2e-4550-8d26-5b431e7997be None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:52.369129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79983d36-6b2e-4550-8d26-5b431e7997be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:52.381179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:52.453087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-79983d36-6b2e-4550-8d26-5b431e7997be None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:52.454653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-79983d36-6b2e-4550-8d26-5b431e7997be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:52.561535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 702/1381] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:52.862363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd82561-0280-4800-bd81-184260c42ad4 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:52.863994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-efd82561-0280-4800-bd81-184260c42ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:53.026236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:53.037998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd82561-0280-4800-bd81-184260c42ad4 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:53.158025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7dcd8-b8fc-4021-b1f1-5f7121b5569c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:53.158940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7dcd8-b8fc-4021-b1f1-5f7121b5569c service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:53.159227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7dcd8-b8fc-4021-b1f1-5f7121b5569c service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:53.159501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7dcd8-b8fc-4021-b1f1-5f7121b5569c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:53.161124 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a0e7dcd8-b8fc-4021-b1f1-5f7121b5569c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:53.259221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7dcd8-b8fc-4021-b1f1-5f7121b5569c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:53.260957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7dcd8-b8fc-4021-b1f1-5f7121b5569c service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:53.272547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:53.281537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 680/1382] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:55.993898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8089d-9ce4-4ecc-9aa4-9f7569f3f2ca None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:55.994453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8089d-9ce4-4ecc-9aa4-9f7569f3f2ca None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:55.994690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8089d-9ce4-4ecc-9aa4-9f7569f3f2ca None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:55.996507 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 703/1383] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:44:56.003439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65a3a-f96d-4a5d-bd63-a750c41b5726 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:56.004710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65a3a-f96d-4a5d-bd63-a750c41b5726 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:56.005155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65a3a-f96d-4a5d-bd63-a750c41b5726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:56.085140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-76e65a3a-f96d-4a5d-bd63-a750c41b5726 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:56.087366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-76e65a3a-f96d-4a5d-bd63-a750c41b5726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:56.201788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 681/1384] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:56.599968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8089d-9ce4-4ecc-9aa4-9f7569f3f2ca None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:56.601538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b5f8089d-9ce4-4ecc-9aa4-9f7569f3f2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:56.696450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:56.772371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8089d-9ce4-4ecc-9aa4-9f7569f3f2ca None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:56.908768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ac3f2-1f22-4149-8ddf-01310c7c046e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:56.909427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac3f2-1f22-4149-8ddf-01310c7c046e service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:56.909710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac3f2-1f22-4149-8ddf-01310c7c046e service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:56.910013 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac3f2-1f22-4149-8ddf-01310c7c046e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:56.911722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6e2ac3f2-1f22-4149-8ddf-01310c7c046e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:56.965625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:57.006733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ac3f2-1f22-4149-8ddf-01310c7c046e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:57.008366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ac3f2-1f22-4149-8ddf-01310c7c046e service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:57.028840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 704/1385] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:59.621358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29481cc6-9965-44c7-a80f-3e830d3e5132 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:59.622624 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29481cc6-9965-44c7-a80f-3e830d3e5132 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:59.622724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29481cc6-9965-44c7-a80f-3e830d3e5132 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:59.624820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 682/1386] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:44:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:44:59.630289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97ce87-8938-449f-ae54-405ab60078ce None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:59.630835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97ce87-8938-449f-ae54-405ab60078ce None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:59.630952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97ce87-8938-449f-ae54-405ab60078ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:59.712457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ba97ce87-8938-449f-ae54-405ab60078ce None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:59.713991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ba97ce87-8938-449f-ae54-405ab60078ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:59.822884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 705/1387] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:00.127339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29481cc6-9965-44c7-a80f-3e830d3e5132 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:00.128867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-29481cc6-9965-44c7-a80f-3e830d3e5132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:00.205649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:00.276025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29481cc6-9965-44c7-a80f-3e830d3e5132 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:00.392358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105789d3-ddf0-431a-ac4c-44f343e5b245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:00.392967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105789d3-ddf0-431a-ac4c-44f343e5b245 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:00.393250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105789d3-ddf0-431a-ac4c-44f343e5b245 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:00.393523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105789d3-ddf0-431a-ac4c-44f343e5b245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:00.395150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-105789d3-ddf0-431a-ac4c-44f343e5b245 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:00.433670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:00.487546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105789d3-ddf0-431a-ac4c-44f343e5b245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:00.489163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105789d3-ddf0-431a-ac4c-44f343e5b245 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:00.507267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 683/1388] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:01.433231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7212ca-169c-48be-9181-9bfbd2e6dd24 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:01.433802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7212ca-169c-48be-9181-9bfbd2e6dd24 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:01.433802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7212ca-169c-48be-9181-9bfbd2e6dd24 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:01.435925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 706/1389] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:01.441035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4544278-3910-4124-8fb7-fcf9c8dc95bf None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:01.441450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4544278-3910-4124-8fb7-fcf9c8dc95bf None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:01.441704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4544278-3910-4124-8fb7-fcf9c8dc95bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:01.507080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-a4544278-3910-4124-8fb7-fcf9c8dc95bf None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:01.508287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a4544278-3910-4124-8fb7-fcf9c8dc95bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:01.607343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 684/1390] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:01.911374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f881d-8d92-48d6-8d6e-5f4bc1048ce1 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:01.911730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f881d-8d92-48d6-8d6e-5f4bc1048ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:01.912081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f881d-8d92-48d6-8d6e-5f4bc1048ce1 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:01.913493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 707/1391] 10.222.0.82 () {54 vars in 967 bytes} [Wed Aug 24 03:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:01.918598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35fa91-3aa5-4789-9292-55f460aa9b13 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:01.919402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35fa91-3aa5-4789-9292-55f460aa9b13 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:01.919684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35fa91-3aa5-4789-9292-55f460aa9b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:01.936164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:02.000496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-ce35fa91-3aa5-4789-9292-55f460aa9b13 None None] MFA Rules not processed for user `4ff3f53ddbb04eb8916b9ccfdb03dd80`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:02.001744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ce35fa91-3aa5-4789-9292-55f460aa9b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:02.104670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 685/1392] 10.222.0.82 () {58 vars in 1064 bytes} [Wed Aug 24 03:45:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:02.113242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f881d-8d92-48d6-8d6e-5f4bc1048ce1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:02.114788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-039f881d-8d92-48d6-8d6e-5f4bc1048ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:02.150625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:02.242456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f881d-8d92-48d6-8d6e-5f4bc1048ce1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:02.323794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab1805-7bbb-4f66-908d-6312b028e022 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ff3f53ddbb04eb8916b9ccfdb03dd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:02.324411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1805-7bbb-4f66-908d-6312b028e022 service glance] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:02.324811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1805-7bbb-4f66-908d-6312b028e022 service glance] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:02.325142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab1805-7bbb-4f66-908d-6312b028e022 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:02.326815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-33ab1805-7bbb-4f66-908d-6312b028e022 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:02.414356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:02.448829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab1805-7bbb-4f66-908d-6312b028e022 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:02.450981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab1805-7bbb-4f66-908d-6312b028e022 service glance] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:02.471117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 708/1393] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:02.600206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8350e50-dfcb-423d-a39a-3c2b7aa7978e None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:02.600765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8350e50-dfcb-423d-a39a-3c2b7aa7978e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:02.600765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8350e50-dfcb-423d-a39a-3c2b7aa7978e None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:02.602082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 686/1394] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:02.607616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1c75-5b80-433b-953c-7693a56a57f2 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:02.607915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1c75-5b80-433b-953c-7693a56a57f2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:02.608195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1c75-5b80-433b-953c-7693a56a57f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:02.693022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-4a3d1c75-5b80-433b-953c-7693a56a57f2 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:02.694687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4a3d1c75-5b80-433b-953c-7693a56a57f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:02.818239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 709/1395] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:03.123775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8350e50-dfcb-423d-a39a-3c2b7aa7978e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:03.125315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d8350e50-dfcb-423d-a39a-3c2b7aa7978e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:03.239551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:03.290527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8350e50-dfcb-423d-a39a-3c2b7aa7978e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:03.411891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c32fd-b9b3-4bf6-b75a-c33dafbb2317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:03.412522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c32fd-b9b3-4bf6-b75a-c33dafbb2317 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:03.412805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c32fd-b9b3-4bf6-b75a-c33dafbb2317 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:03.413079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c32fd-b9b3-4bf6-b75a-c33dafbb2317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:03.414736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ea2c32fd-b9b3-4bf6-b75a-c33dafbb2317 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:03.438189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37fe31-ac53-46bd-89e8-58e2d0cab1bd None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:03.438478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37fe31-ac53-46bd-89e8-58e2d0cab1bd None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:03.438801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37fe31-ac53-46bd-89e8-58e2d0cab1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:03.484342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:03.486223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:03.506047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-be37fe31-ac53-46bd-89e8-58e2d0cab1bd None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:03.506267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c32fd-b9b3-4bf6-b75a-c33dafbb2317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:03.507305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-be37fe31-ac53-46bd-89e8-58e2d0cab1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:03.507804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c32fd-b9b3-4bf6-b75a-c33dafbb2317 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:03.527199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 687/1396] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:03.596714 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 710/1397] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:45:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:45:03.643909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad1f7-392e-46b7-a9e0-29c863e005cc None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:03.644445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad1f7-392e-46b7-a9e0-29c863e005cc None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:03.644445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad1f7-392e-46b7-a9e0-29c863e005cc None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:03.645824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 688/1398] 10.222.0.82 () {54 vars in 943 bytes} [Wed Aug 24 03:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:03.653326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ed2c5-95b2-470d-8de6-4412b7e735c2 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:03.653626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ed2c5-95b2-470d-8de6-4412b7e735c2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:03.654041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ed2c5-95b2-470d-8de6-4412b7e735c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:03.728579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9f8ed2c5-95b2-470d-8de6-4412b7e735c2 None None] MFA Rules not processed for user `ad67f6ace29a4591bf2b8b4f119b3323`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:03.729793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9f8ed2c5-95b2-470d-8de6-4412b7e735c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:03.798461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:03.850086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 711/1399] 10.222.0.82 () {58 vars in 1040 bytes} [Wed Aug 24 03:45:03 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:03.860713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ad1f7-392e-46b7-a9e0-29c863e005cc None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:03.862346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0e3ad1f7-392e-46b7-a9e0-29c863e005cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:04.021365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ad1f7-392e-46b7-a9e0-29c863e005cc None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:04.078031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:04.127933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edf632-965d-4961-a65d-123ed1601590 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:04.128546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edf632-965d-4961-a65d-123ed1601590 service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:04.128828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edf632-965d-4961-a65d-123ed1601590 service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:04.129100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edf632-965d-4961-a65d-123ed1601590 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:04.130777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e1edf632-965d-4961-a65d-123ed1601590 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:04.214895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edf632-965d-4961-a65d-123ed1601590 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:04.216569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edf632-965d-4961-a65d-123ed1601590 service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:04.224904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 689/1400] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:45:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:45:05.672155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630dbe0-5404-4742-bbcd-0480bfa794dc None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:05.672689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630dbe0-5404-4742-bbcd-0480bfa794dc None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:05.672689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630dbe0-5404-4742-bbcd-0480bfa794dc None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:05.674595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 712/1401] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:05.680915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a7529-ff12-4c5e-9114-b87aa99f9cb6 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:05.680915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a7529-ff12-4c5e-9114-b87aa99f9cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:05.680915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a7529-ff12-4c5e-9114-b87aa99f9cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:05.758173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7a5a7529-ff12-4c5e-9114-b87aa99f9cb6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:05.759550 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7a5a7529-ff12-4c5e-9114-b87aa99f9cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:05.808943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:05.896171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 690/1402] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:45:06.222917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630dbe0-5404-4742-bbcd-0480bfa794dc None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:06.222917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4630dbe0-5404-4742-bbcd-0480bfa794dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:06.398000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630dbe0-5404-4742-bbcd-0480bfa794dc None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:06.451951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:06.556941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e54c05-10a0-4df6-92d3-530cc42c8f39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:06.557603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54c05-10a0-4df6-92d3-530cc42c8f39 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:06.557896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54c05-10a0-4df6-92d3-530cc42c8f39 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:06.558307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e54c05-10a0-4df6-92d3-530cc42c8f39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:06.561323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-47e54c05-10a0-4df6-92d3-530cc42c8f39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:06.705834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e54c05-10a0-4df6-92d3-530cc42c8f39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:06.708225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e54c05-10a0-4df6-92d3-530cc42c8f39 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:06.739835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 713/1403] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:45:09.147952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625e163-8518-4026-ad07-c1fe2563b55c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:09.148516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625e163-8518-4026-ad07-c1fe2563b55c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:09.148516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625e163-8518-4026-ad07-c1fe2563b55c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:09.149899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 691/1404] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:09.155964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8191f-aa4f-4b48-b3b2-9ccd4a491652 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:09.156410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8191f-aa4f-4b48-b3b2-9ccd4a491652 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:09.156515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8191f-aa4f-4b48-b3b2-9ccd4a491652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:09.221671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:09.229828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-03a8191f-aa4f-4b48-b3b2-9ccd4a491652 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:09.231815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-03a8191f-aa4f-4b48-b3b2-9ccd4a491652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:09.343709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 714/1405] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:45:09.648943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625e163-8518-4026-ad07-c1fe2563b55c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:09.650589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b625e163-8518-4026-ad07-c1fe2563b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:09.813022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625e163-8518-4026-ad07-c1fe2563b55c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:09.841925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:09.947730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f026c-f106-40ef-8786-48288e8650bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:09.948704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f026c-f106-40ef-8786-48288e8650bf service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:09.948988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f026c-f106-40ef-8786-48288e8650bf service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:09.949264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f026c-f106-40ef-8786-48288e8650bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:09.951966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-220f026c-f106-40ef-8786-48288e8650bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:10.061322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f026c-f106-40ef-8786-48288e8650bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:10.063361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f026c-f106-40ef-8786-48288e8650bf service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:10.085289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 692/1406] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:12.217929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427a3f7-57be-4ccb-932c-431961b246ee None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:12.218718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427a3f7-57be-4ccb-932c-431961b246ee None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:12.218718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427a3f7-57be-4ccb-932c-431961b246ee None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:12.226789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d60c1-661d-492d-a91f-3013ee8bb7d5 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:12.227394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d60c1-661d-492d-a91f-3013ee8bb7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:12.227394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d60c1-661d-492d-a91f-3013ee8bb7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:12.233004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 715/1407] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:45:12 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:45:12.269950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:12.308271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7c7d60c1-661d-492d-a91f-3013ee8bb7d5 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:12.310280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7c7d60c1-661d-492d-a91f-3013ee8bb7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:12.448041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 693/1408] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:12.758086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427a3f7-57be-4ccb-932c-431961b246ee None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:12.759648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e427a3f7-57be-4ccb-932c-431961b246ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:12.901744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:12.934300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427a3f7-57be-4ccb-932c-431961b246ee None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:13.071263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9170aa-1036-405d-8284-410292146ec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:13.071889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9170aa-1036-405d-8284-410292146ec9 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:13.072173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9170aa-1036-405d-8284-410292146ec9 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:13.072450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9170aa-1036-405d-8284-410292146ec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:13.075316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4c9170aa-1036-405d-8284-410292146ec9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:13.180746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:13.185488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9170aa-1036-405d-8284-410292146ec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:13.186990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9170aa-1036-405d-8284-410292146ec9 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:13.208032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 716/1409] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:22.770290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=52 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:23.222421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=7 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:28.679619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f6949-cea2-4544-aa77-0c257cf6cda7 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:28.680209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f6949-cea2-4544-aa77-0c257cf6cda7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:28.680209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f6949-cea2-4544-aa77-0c257cf6cda7 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:28.681537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 694/1410] 10.222.0.82 () {54 vars in 918 bytes} [Wed Aug 24 03:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:45:28.687247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9956c-d44d-4eab-b57c-8455ae3c8de4 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:28.688014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9956c-d44d-4eab-b57c-8455ae3c8de4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:28.688241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9956c-d44d-4eab-b57c-8455ae3c8de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:28.766266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-38a9956c-d44d-4eab-b57c-8455ae3c8de4 None None] MFA Rules not processed for user `cff23ae056ff452588cd0b6520e96962`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:28.767540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-38a9956c-d44d-4eab-b57c-8455ae3c8de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:28.888793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 717/1411] 10.222.0.82 () {58 vars in 1015 bytes} [Wed Aug 24 03:45:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:28.905083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbf1ec-29c5-4023-93e2-7cc8b72d7d70 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:28.905369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbf1ec-29c5-4023-93e2-7cc8b72d7d70 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:28.905645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbf1ec-29c5-4023-93e2-7cc8b72d7d70 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:28.907029 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 695/1412] 10.222.0.82 () {54 vars in 918 bytes} [Wed Aug 24 03:45:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:28.912481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97570a1-8929-4764-94ec-2f0cda0a7f0b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:28.912770 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97570a1-8929-4764-94ec-2f0cda0a7f0b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:28.913054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97570a1-8929-4764-94ec-2f0cda0a7f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:29.111665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:29.114350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a97570a1-8929-4764-94ec-2f0cda0a7f0b None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:29.115625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a97570a1-8929-4764-94ec-2f0cda0a7f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:29.232248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 718/1413] 10.222.0.82 () {58 vars in 1015 bytes} [Wed Aug 24 03:45:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:31.671935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b596ed7-f165-4362-bfd3-1ec89e375919 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:31.672487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b596ed7-f165-4362-bfd3-1ec89e375919 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:31.672487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b596ed7-f165-4362-bfd3-1ec89e375919 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:31.673841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:31.679466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ad182-acf1-498c-8932-7bbc62e4bb15 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:31.679892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ad182-acf1-498c-8932-7bbc62e4bb15 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:31.680020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ad182-acf1-498c-8932-7bbc62e4bb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:31.747365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-de6ad182-acf1-498c-8932-7bbc62e4bb15 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:31.748599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-de6ad182-acf1-498c-8932-7bbc62e4bb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:31.804319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:31.851092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:32.162690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b596ed7-f165-4362-bfd3-1ec89e375919 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:32.162690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2b596ed7-f165-4362-bfd3-1ec89e375919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:32.314986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b596ed7-f165-4362-bfd3-1ec89e375919 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:32.387432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:32.432596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c124e-ea81-4d70-881d-4be0e2c23012 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:32.434333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c124e-ea81-4d70-881d-4be0e2c23012 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:32.434333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c124e-ea81-4d70-881d-4be0e2c23012 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:32.434333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c124e-ea81-4d70-881d-4be0e2c23012 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:32.436339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-066c124e-ea81-4d70-881d-4be0e2c23012 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:32.526621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c124e-ea81-4d70-881d-4be0e2c23012 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:32.529106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c124e-ea81-4d70-881d-4be0e2c23012 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:32.548618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 697/1416] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:33.448352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2cc01-ad47-4bd1-b331-5ce023b54f78 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:33.448916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2cc01-ad47-4bd1-b331-5ce023b54f78 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:33.448916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2cc01-ad47-4bd1-b331-5ce023b54f78 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:33.451070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:33.455933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feea71d-41b8-41e2-876c-66eddfc480e2 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:33.456385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feea71d-41b8-41e2-876c-66eddfc480e2 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:33.456503 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feea71d-41b8-41e2-876c-66eddfc480e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:33.537672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-0feea71d-41b8-41e2-876c-66eddfc480e2 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:33.538918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0feea71d-41b8-41e2-876c-66eddfc480e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:33.566887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:33.639942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:33.798153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2cc01-ad47-4bd1-b331-5ce023b54f78 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:33.799697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-aba2cc01-ad47-4bd1-b331-5ce023b54f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:33.948331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2cc01-ad47-4bd1-b331-5ce023b54f78 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:33.988806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:34.070463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a410ed8-2877-4c8d-a339-a6732994f52b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:34.071104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a410ed8-2877-4c8d-a339-a6732994f52b service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:34.071386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a410ed8-2877-4c8d-a339-a6732994f52b service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:34.071661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a410ed8-2877-4c8d-a339-a6732994f52b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:34.073274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2a410ed8-2877-4c8d-a339-a6732994f52b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:34.164745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a410ed8-2877-4c8d-a339-a6732994f52b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:34.166356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a410ed8-2877-4c8d-a339-a6732994f52b service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:34.185781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 721/1419] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:35.076808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc09a2-6859-46b1-98d7-4713741fda6d None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:35.077389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc09a2-6859-46b1-98d7-4713741fda6d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:35.077389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc09a2-6859-46b1-98d7-4713741fda6d None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:35.078706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:35.083593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8e2fa-671d-420c-8bd9-64b60aa4cc70 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:35.084028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8e2fa-671d-420c-8bd9-64b60aa4cc70 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:35.084152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8e2fa-671d-420c-8bd9-64b60aa4cc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:35.142067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:35.164175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-c9f8e2fa-671d-420c-8bd9-64b60aa4cc70 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:35.165396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c9f8e2fa-671d-420c-8bd9-64b60aa4cc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:35.269306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:35.421454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc09a2-6859-46b1-98d7-4713741fda6d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:35.423011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0bfc09a2-6859-46b1-98d7-4713741fda6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:35.585370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc09a2-6859-46b1-98d7-4713741fda6d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:35.598597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:35.714992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e81511-11a1-4945-9899-4fb7d183bc13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:35.715706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e81511-11a1-4945-9899-4fb7d183bc13 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:35.716025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e81511-11a1-4945-9899-4fb7d183bc13 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:35.716305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e81511-11a1-4945-9899-4fb7d183bc13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:35.717923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a0e81511-11a1-4945-9899-4fb7d183bc13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:35.812173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e81511-11a1-4945-9899-4fb7d183bc13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:35.813793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e81511-11a1-4945-9899-4fb7d183bc13 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:35.832494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 700/1422] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:36.714313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b8029-3904-45a0-b643-9200c68a98f9 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:36.714891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b8029-3904-45a0-b643-9200c68a98f9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:36.715021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b8029-3904-45a0-b643-9200c68a98f9 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:36.717522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:45:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:36.722839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e592037-a315-4c02-adc9-fd029a7c1c68 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:36.723401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e592037-a315-4c02-adc9-fd029a7c1c68 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:36.723527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e592037-a315-4c02-adc9-fd029a7c1c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:36.744849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:36.804406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6e592037-a315-4c02-adc9-fd029a7c1c68 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:36.805551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6e592037-a315-4c02-adc9-fd029a7c1c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:36.905710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:37.056055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b8029-3904-45a0-b643-9200c68a98f9 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:37.057487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c81b8029-3904-45a0-b643-9200c68a98f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:37.167035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:37.202981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b8029-3904-45a0-b643-9200c68a98f9 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:37.316474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a8b5d-0a83-4aa5-a4bd-b0259cb73ace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:37.316975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8b5d-0a83-4aa5-a4bd-b0259cb73ace service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:37.317251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8b5d-0a83-4aa5-a4bd-b0259cb73ace service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:37.317517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8b5d-0a83-4aa5-a4bd-b0259cb73ace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:37.319131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-989a8b5d-0a83-4aa5-a4bd-b0259cb73ace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:37.394028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:37.408641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a8b5d-0a83-4aa5-a4bd-b0259cb73ace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:37.410254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a8b5d-0a83-4aa5-a4bd-b0259cb73ace service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:37.429359 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 724/1425] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:38.348511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67537e-6325-4ca5-928b-0745618de9b0 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:38.349046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67537e-6325-4ca5-928b-0745618de9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:38.349251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67537e-6325-4ca5-928b-0745618de9b0 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:38.351377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:45:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:38.357897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcea801-3453-460b-a109-02216943c8ca None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:38.358580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcea801-3453-460b-a109-02216943c8ca None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:38.358580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcea801-3453-460b-a109-02216943c8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:38.444845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5dcea801-3453-460b-a109-02216943c8ca None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:38.446082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5dcea801-3453-460b-a109-02216943c8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:38.549288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:45:38.706250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67537e-6325-4ca5-928b-0745618de9b0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:38.707870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ed67537e-6325-4ca5-928b-0745618de9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:38.804796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:38.872293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67537e-6325-4ca5-928b-0745618de9b0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:38.990746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78158982-498d-48a5-b865-62e2f35ad410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:38.991273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78158982-498d-48a5-b865-62e2f35ad410 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:38.991556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78158982-498d-48a5-b865-62e2f35ad410 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:38.991833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78158982-498d-48a5-b865-62e2f35ad410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:38.993520 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-78158982-498d-48a5-b865-62e2f35ad410 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:39.047495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:39.093442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78158982-498d-48a5-b865-62e2f35ad410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:39.095148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78158982-498d-48a5-b865-62e2f35ad410 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:39.113427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 703/1428] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:39.988495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155da6e-81f6-43f5-9f40-4ee212afb2d5 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:39.989062 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155da6e-81f6-43f5-9f40-4ee212afb2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:39.989062 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155da6e-81f6-43f5-9f40-4ee212afb2d5 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:39.990440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:39.995568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a40649-f2a5-4381-955d-66a30d7aced4 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:39.999524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a40649-f2a5-4381-955d-66a30d7aced4 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:39.999524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a40649-f2a5-4381-955d-66a30d7aced4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:40.079096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d3a40649-f2a5-4381-955d-66a30d7aced4 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:40.080301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d3a40649-f2a5-4381-955d-66a30d7aced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:40.178217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:40.337198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155da6e-81f6-43f5-9f40-4ee212afb2d5 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:40.338652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3155da6e-81f6-43f5-9f40-4ee212afb2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:40.388939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:40.483804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155da6e-81f6-43f5-9f40-4ee212afb2d5 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:40.593782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df84ca-b896-4d2c-82d0-7528e48aa3ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:40.594309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df84ca-b896-4d2c-82d0-7528e48aa3ae service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:40.594615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df84ca-b896-4d2c-82d0-7528e48aa3ae service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:40.594885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df84ca-b896-4d2c-82d0-7528e48aa3ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:40.596464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a8df84ca-b896-4d2c-82d0-7528e48aa3ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:40.603226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:40.688682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df84ca-b896-4d2c-82d0-7528e48aa3ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:40.690294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df84ca-b896-4d2c-82d0-7528e48aa3ae service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:40.708902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 727/1431] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:41.624711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d052395-8710-4394-9fce-0cc6a217c43c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:41.625270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d052395-8710-4394-9fce-0cc6a217c43c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:41.625270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d052395-8710-4394-9fce-0cc6a217c43c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:41.626667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:41.631727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31d2e7-ec75-4595-b8ca-d5103c33cef1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:41.632137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31d2e7-ec75-4595-b8ca-d5103c33cef1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:41.632276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31d2e7-ec75-4595-b8ca-d5103c33cef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:41.707695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9a31d2e7-ec75-4595-b8ca-d5103c33cef1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:41.709010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9a31d2e7-ec75-4595-b8ca-d5103c33cef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:41.779760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:41.811717 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:41.965483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d052395-8710-4394-9fce-0cc6a217c43c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:41.966985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4d052395-8710-4394-9fce-0cc6a217c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:41.984187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:42.119739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d052395-8710-4394-9fce-0cc6a217c43c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:42.209391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:42.235568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8591eb-3adf-4aba-90d4-5f95d01e8195 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:42.236110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8591eb-3adf-4aba-90d4-5f95d01e8195 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:42.236393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8591eb-3adf-4aba-90d4-5f95d01e8195 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:42.236668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8591eb-3adf-4aba-90d4-5f95d01e8195 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:42.238275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8d8591eb-3adf-4aba-90d4-5f95d01e8195 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:42.327785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8591eb-3adf-4aba-90d4-5f95d01e8195 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:42.329415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8591eb-3adf-4aba-90d4-5f95d01e8195 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:42.347154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 706/1434] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:43.241111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ac420-272b-40fb-9ec8-b6ea876f3064 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:43.241648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ac420-272b-40fb-9ec8-b6ea876f3064 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:43.241778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ac420-272b-40fb-9ec8-b6ea876f3064 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:43.243228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:43.248825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67449e9c-492d-4be6-8242-396a7699c0f4 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:43.249235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67449e9c-492d-4be6-8242-396a7699c0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:43.249377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67449e9c-492d-4be6-8242-396a7699c0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:43.325619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-67449e9c-492d-4be6-8242-396a7699c0f4 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:43.326770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-67449e9c-492d-4be6-8242-396a7699c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:43.373053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:43.430588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:43.587377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ac420-272b-40fb-9ec8-b6ea876f3064 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:43.588832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b14ac420-272b-40fb-9ec8-b6ea876f3064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:43.732898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ac420-272b-40fb-9ec8-b6ea876f3064 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:43.788596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:43.847707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4cc3f-d40d-40f8-b91f-0e0d5613c5de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:43.848239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4cc3f-d40d-40f8-b91f-0e0d5613c5de service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:43.848522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4cc3f-d40d-40f8-b91f-0e0d5613c5de service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:43.848798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4cc3f-d40d-40f8-b91f-0e0d5613c5de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:43.850407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-22a4cc3f-d40d-40f8-b91f-0e0d5613c5de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:43.946077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4cc3f-d40d-40f8-b91f-0e0d5613c5de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:43.947764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4cc3f-d40d-40f8-b91f-0e0d5613c5de service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:43.966924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 730/1437] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:44.861481 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5280c-ccc6-448a-b452-ceee39be2fa2 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:44.862017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5280c-ccc6-448a-b452-ceee39be2fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:44.862017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5280c-ccc6-448a-b452-ceee39be2fa2 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:44.863432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:44.868485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714a441-8797-4045-9004-c4ed75b123bb None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:44.868940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714a441-8797-4045-9004-c4ed75b123bb None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:44.869076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714a441-8797-4045-9004-c4ed75b123bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:44.947520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-7714a441-8797-4045-9004-c4ed75b123bb None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:44.948774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7714a441-8797-4045-9004-c4ed75b123bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:44.951475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:45.049766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:45.205613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5280c-ccc6-448a-b452-ceee39be2fa2 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:45.207210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-27e5280c-ccc6-448a-b452-ceee39be2fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:45.355111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5280c-ccc6-448a-b452-ceee39be2fa2 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:45.381466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:45.468945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407ed1c-bacf-4a63-ab2c-254aa21de4d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:45.469555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407ed1c-bacf-4a63-ab2c-254aa21de4d0 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:45.469835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407ed1c-bacf-4a63-ab2c-254aa21de4d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:45.470103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407ed1c-bacf-4a63-ab2c-254aa21de4d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:45.471724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3407ed1c-bacf-4a63-ab2c-254aa21de4d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:45.559917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407ed1c-bacf-4a63-ab2c-254aa21de4d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:45.561758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407ed1c-bacf-4a63-ab2c-254aa21de4d0 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:45.580517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 709/1440] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:47.426036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454c2ad-675b-4a44-b763-ee0946fbcef4 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:47.426586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454c2ad-675b-4a44-b763-ee0946fbcef4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:47.426688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454c2ad-675b-4a44-b763-ee0946fbcef4 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:47.427996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:47.433497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e6e5a-3928-4bd9-b65b-776901aadace None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:47.433907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e6e5a-3928-4bd9-b65b-776901aadace None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:47.434075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e6e5a-3928-4bd9-b65b-776901aadace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:47.477450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:47.503322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b80e6e5a-3928-4bd9-b65b-776901aadace None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:47.504543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b80e6e5a-3928-4bd9-b65b-776901aadace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:47.606534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:47.895908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454c2ad-675b-4a44-b763-ee0946fbcef4 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:47.897496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6454c2ad-675b-4a44-b763-ee0946fbcef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:48.005853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454c2ad-675b-4a44-b763-ee0946fbcef4 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:48.040530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:48.082802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33a084-0cd9-4f1f-9309-770f59c5ab42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:48.083418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a084-0cd9-4f1f-9309-770f59c5ab42 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:48.083699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a084-0cd9-4f1f-9309-770f59c5ab42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:48.083972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a084-0cd9-4f1f-9309-770f59c5ab42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:48.085574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4c33a084-0cd9-4f1f-9309-770f59c5ab42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:48.198201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33a084-0cd9-4f1f-9309-770f59c5ab42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:48.201582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33a084-0cd9-4f1f-9309-770f59c5ab42 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:48.226966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 733/1443] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:45:50.301310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4303c-a362-4edc-a096-20e2cba4b15b None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:50.301852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4303c-a362-4edc-a096-20e2cba4b15b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:50.301852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4303c-a362-4edc-a096-20e2cba4b15b None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:50.303187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:50.308385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4ea49-d86c-49b1-bab3-072b89b3b455 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:50.308778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4ea49-d86c-49b1-bab3-072b89b3b455 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:50.308936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4ea49-d86c-49b1-bab3-072b89b3b455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:50.375120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e0e4ea49-d86c-49b1-bab3-072b89b3b455 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:50.376353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e0e4ea49-d86c-49b1-bab3-072b89b3b455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:50.389817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:50.486300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:50.773763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4303c-a362-4edc-a096-20e2cba4b15b None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:50.775405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3df4303c-a362-4edc-a096-20e2cba4b15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:50.900502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4303c-a362-4edc-a096-20e2cba4b15b None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:50.913597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:50.974958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733be137-ebb9-4b34-9ded-39be065d8f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:50.975566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733be137-ebb9-4b34-9ded-39be065d8f95 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:50.975852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733be137-ebb9-4b34-9ded-39be065d8f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:50.976128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733be137-ebb9-4b34-9ded-39be065d8f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:50.977742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-733be137-ebb9-4b34-9ded-39be065d8f95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:51.089814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733be137-ebb9-4b34-9ded-39be065d8f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:51.091620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733be137-ebb9-4b34-9ded-39be065d8f95 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:51.109410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 712/1446] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:53.433042 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6d0ac-299e-4c91-8440-60c99a3b039b None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:53.433620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6d0ac-299e-4c91-8440-60c99a3b039b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:53.433620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6d0ac-299e-4c91-8440-60c99a3b039b None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:53.435239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:53.440544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaf4fc-d48e-4baf-9b3d-d86700147dc7 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:53.441061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaf4fc-d48e-4baf-9b3d-d86700147dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:53.441061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaf4fc-d48e-4baf-9b3d-d86700147dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:53.489917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:53.507545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-0beaf4fc-d48e-4baf-9b3d-d86700147dc7 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:53.508771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0beaf4fc-d48e-4baf-9b3d-d86700147dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:53.609041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:45:53.909906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6d0ac-299e-4c91-8440-60c99a3b039b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:53.911489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ffb6d0ac-299e-4c91-8440-60c99a3b039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:54.037812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6d0ac-299e-4c91-8440-60c99a3b039b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:54.081258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:54.117403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dd6ca-b10e-45e9-9b52-7baa1dc3f0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:54.118017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dd6ca-b10e-45e9-9b52-7baa1dc3f0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:54.118298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dd6ca-b10e-45e9-9b52-7baa1dc3f0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:54.118602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dd6ca-b10e-45e9-9b52-7baa1dc3f0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:54.120220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f10dd6ca-b10e-45e9-9b52-7baa1dc3f0a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:54.231872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dd6ca-b10e-45e9-9b52-7baa1dc3f0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:54.233347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dd6ca-b10e-45e9-9b52-7baa1dc3f0a6 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:54.252240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 736/1449] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:56.323817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27bd0c-093a-4ccf-805c-6edd46726965 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:56.324350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27bd0c-093a-4ccf-805c-6edd46726965 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:56.324350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27bd0c-093a-4ccf-805c-6edd46726965 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:56.325861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:56.331313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c39e1-d5db-4fdb-9336-6369f65f7531 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:56.331738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c39e1-d5db-4fdb-9336-6369f65f7531 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:56.331871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c39e1-d5db-4fdb-9336-6369f65f7531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:56.398617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-549c39e1-d5db-4fdb-9336-6369f65f7531 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:56.399886 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-549c39e1-d5db-4fdb-9336-6369f65f7531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:56.422267 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:56.501690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:56.830859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27bd0c-093a-4ccf-805c-6edd46726965 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:56.832470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cb27bd0c-093a-4ccf-805c-6edd46726965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:56.977435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27bd0c-093a-4ccf-805c-6edd46726965 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:57.002930 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:57.085543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e70b65-fd87-41d1-84de-d53ce7946b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:57.086332 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e70b65-fd87-41d1-84de-d53ce7946b96 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:57.086782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e70b65-fd87-41d1-84de-d53ce7946b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:57.087220 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e70b65-fd87-41d1-84de-d53ce7946b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:57.089878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b9e70b65-fd87-41d1-84de-d53ce7946b96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:57.243903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e70b65-fd87-41d1-84de-d53ce7946b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:57.246310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e70b65-fd87-41d1-84de-d53ce7946b96 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:57.275595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 715/1452] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:59.321182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed227bca-6f59-49eb-9f17-85e7e8cb9cb8 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:59.321711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed227bca-6f59-49eb-9f17-85e7e8cb9cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:59.321711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed227bca-6f59-49eb-9f17-85e7e8cb9cb8 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:59.323045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:45:59.328752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ba6f5-aecc-4e21-a08b-d41d1c9db16a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:59.329257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ba6f5-aecc-4e21-a08b-d41d1c9db16a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:59.329369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ba6f5-aecc-4e21-a08b-d41d1c9db16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:59.394782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:59.397522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-af4ba6f5-aecc-4e21-a08b-d41d1c9db16a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:59.398782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-af4ba6f5-aecc-4e21-a08b-d41d1c9db16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:59.497681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:59.783220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed227bca-6f59-49eb-9f17-85e7e8cb9cb8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:59.784796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ed227bca-6f59-49eb-9f17-85e7e8cb9cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:59.894389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed227bca-6f59-49eb-9f17-85e7e8cb9cb8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:59.947634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:59.973970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360279af-f9db-479d-8260-464c9638fcd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:59.974624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360279af-f9db-479d-8260-464c9638fcd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:59.974905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360279af-f9db-479d-8260-464c9638fcd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:59.975181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360279af-f9db-479d-8260-464c9638fcd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:59.976834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-360279af-f9db-479d-8260-464c9638fcd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:00.095913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360279af-f9db-479d-8260-464c9638fcd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:00.097283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360279af-f9db-479d-8260-464c9638fcd9 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:00.116278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 739/1455] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:02.200425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfafaf8-b4f6-48f3-b103-3b9c0d5413e5 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:02.201096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfafaf8-b4f6-48f3-b103-3b9c0d5413e5 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:02.201255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfafaf8-b4f6-48f3-b103-3b9c0d5413e5 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:02.202618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:02.208651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339c179-26d3-4fbc-8571-10e5d9a7480f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:02.209161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339c179-26d3-4fbc-8571-10e5d9a7480f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:02.209261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339c179-26d3-4fbc-8571-10e5d9a7480f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:02.286344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-4339c179-26d3-4fbc-8571-10e5d9a7480f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:02.287776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4339c179-26d3-4fbc-8571-10e5d9a7480f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:02.318632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:02.389377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:02.679624 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfafaf8-b4f6-48f3-b103-3b9c0d5413e5 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:02.681178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7dfafaf8-b4f6-48f3-b103-3b9c0d5413e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:02.797683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfafaf8-b4f6-48f3-b103-3b9c0d5413e5 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:02.833284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:02.881856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f3ef7-2e3b-4852-8db3-0f086de7215d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:02.882469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f3ef7-2e3b-4852-8db3-0f086de7215d service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:02.882788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f3ef7-2e3b-4852-8db3-0f086de7215d service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:02.883061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f3ef7-2e3b-4852-8db3-0f086de7215d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:02.884692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bc7f3ef7-2e3b-4852-8db3-0f086de7215d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:03.014890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f3ef7-2e3b-4852-8db3-0f086de7215d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:03.016272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f3ef7-2e3b-4852-8db3-0f086de7215d service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:03.035023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 718/1458] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:05.039477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3588062-12f6-43fc-9754-5b6cca5e34a0 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:05.039988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3588062-12f6-43fc-9754-5b6cca5e34a0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:05.040105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3588062-12f6-43fc-9754-5b6cca5e34a0 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:05.041451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:05.048001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e4ab5-e5f7-4cb4-b140-12d0e2be9a06 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:05.048444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e4ab5-e5f7-4cb4-b140-12d0e2be9a06 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:05.048596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e4ab5-e5f7-4cb4-b140-12d0e2be9a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:05.116604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-4b7e4ab5-e5f7-4cb4-b140-12d0e2be9a06 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:05.117864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4b7e4ab5-e5f7-4cb4-b140-12d0e2be9a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:05.126296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:05.215092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:05.495918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3588062-12f6-43fc-9754-5b6cca5e34a0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:05.498127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f3588062-12f6-43fc-9754-5b6cca5e34a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:05.603313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3588062-12f6-43fc-9754-5b6cca5e34a0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:05.661928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:05.681239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1abac-3e50-491f-ade7-cdae6632e624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:05.687782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1abac-3e50-491f-ade7-cdae6632e624 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:05.687782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1abac-3e50-491f-ade7-cdae6632e624 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:05.687782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1abac-3e50-491f-ade7-cdae6632e624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:05.687782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3ca1abac-3e50-491f-ade7-cdae6632e624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:05.794640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1abac-3e50-491f-ade7-cdae6632e624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:05.796179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1abac-3e50-491f-ade7-cdae6632e624 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:05.815076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 742/1461] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:07.805027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75371d7f-f835-4d89-a004-cb86286314d1 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:07.805609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75371d7f-f835-4d89-a004-cb86286314d1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:07.805609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75371d7f-f835-4d89-a004-cb86286314d1 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:07.806950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:07.813431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f1012-85b6-4211-83f1-dc81bc362e82 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:07.813913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f1012-85b6-4211-83f1-dc81bc362e82 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:07.814035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f1012-85b6-4211-83f1-dc81bc362e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:07.883251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-da6f1012-85b6-4211-83f1-dc81bc362e82 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:07.884637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-da6f1012-85b6-4211-83f1-dc81bc362e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:07.926246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:07.988356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:08.287035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75371d7f-f835-4d89-a004-cb86286314d1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:08.288863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-75371d7f-f835-4d89-a004-cb86286314d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:08.397167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75371d7f-f835-4d89-a004-cb86286314d1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:08.437223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:08.473058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7becf3-50d7-4b02-bdc8-8cfcd19cd83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:08.473667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7becf3-50d7-4b02-bdc8-8cfcd19cd83d service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:08.473951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7becf3-50d7-4b02-bdc8-8cfcd19cd83d service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:08.474229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7becf3-50d7-4b02-bdc8-8cfcd19cd83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:08.475876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bc7becf3-50d7-4b02-bdc8-8cfcd19cd83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:08.717886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7becf3-50d7-4b02-bdc8-8cfcd19cd83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:08.719386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7becf3-50d7-4b02-bdc8-8cfcd19cd83d service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:08.738198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 721/1464] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:10.698018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e2c9-0e48-40d6-91f6-d2248e366268 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:10.698704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e2c9-0e48-40d6-91f6-d2248e366268 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:10.698704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e2c9-0e48-40d6-91f6-d2248e366268 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:10.700007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:10.707458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30249338-3696-456d-a779-7eaad3e55d9c None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:10.707956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30249338-3696-456d-a779-7eaad3e55d9c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:10.708071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30249338-3696-456d-a779-7eaad3e55d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:10.775914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-30249338-3696-456d-a779-7eaad3e55d9c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:10.777161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-30249338-3696-456d-a779-7eaad3e55d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:10.796932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:10.883103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:46:11.174200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3e2c9-0e48-40d6-91f6-d2248e366268 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:11.175799 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-dfd3e2c9-0e48-40d6-91f6-d2248e366268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:11.301307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3e2c9-0e48-40d6-91f6-d2248e366268 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:11.368896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:11.375941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9c547-7987-43b5-8b2d-12e7143852ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:11.376549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9c547-7987-43b5-8b2d-12e7143852ca service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:11.376828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9c547-7987-43b5-8b2d-12e7143852ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:11.377101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9c547-7987-43b5-8b2d-12e7143852ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:11.378715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-79d9c547-7987-43b5-8b2d-12e7143852ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:11.495287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9c547-7987-43b5-8b2d-12e7143852ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:11.496749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9c547-7987-43b5-8b2d-12e7143852ca service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:11.515949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 745/1467] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:13.458936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738366-8bbd-4f57-a245-967720011f14 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:13.459478 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738366-8bbd-4f57-a245-967720011f14 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:13.459478 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43738366-8bbd-4f57-a245-967720011f14 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:13.460861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:13.466391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045d2af-a7c5-405a-ace8-8e5e20ab68b8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:13.466961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045d2af-a7c5-405a-ace8-8e5e20ab68b8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:13.467151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045d2af-a7c5-405a-ace8-8e5e20ab68b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:13.532677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e045d2af-a7c5-405a-ace8-8e5e20ab68b8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:13.533890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e045d2af-a7c5-405a-ace8-8e5e20ab68b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:13.582765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:13.631105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:13.921666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43738366-8bbd-4f57-a245-967720011f14 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:13.923253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-43738366-8bbd-4f57-a245-967720011f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:14.029502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43738366-8bbd-4f57-a245-967720011f14 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:14.078213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:14.104130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c54ba-1095-4c83-b26d-304a05e5583d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:14.104833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c54ba-1095-4c83-b26d-304a05e5583d service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:14.105115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c54ba-1095-4c83-b26d-304a05e5583d service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:14.105387 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c54ba-1095-4c83-b26d-304a05e5583d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:14.107761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-419c54ba-1095-4c83-b26d-304a05e5583d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:14.216552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c54ba-1095-4c83-b26d-304a05e5583d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:14.217930 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c54ba-1095-4c83-b26d-304a05e5583d service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:14.237484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 724/1470] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:23.629238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=22 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:24.235194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=66 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:34.239927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d091d-e8af-4603-b3c3-a1decc23f971 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:34.240523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d091d-e8af-4603-b3c3-a1decc23f971 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:34.240523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d091d-e8af-4603-b3c3-a1decc23f971 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:34.241839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 747/1471] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:34.247204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff51f6-1ddd-42ff-8367-a1d09311e9a6 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:34.247730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff51f6-1ddd-42ff-8367-a1d09311e9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:34.248065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff51f6-1ddd-42ff-8367-a1d09311e9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:34.315792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-a4ff51f6-1ddd-42ff-8367-a1d09311e9a6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:34.317002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a4ff51f6-1ddd-42ff-8367-a1d09311e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:34.413895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 725/1472] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:34.714306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a56f0-7aab-47e9-9edc-10f923115f2b None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:34.714636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a56f0-7aab-47e9-9edc-10f923115f2b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:34.714920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a56f0-7aab-47e9-9edc-10f923115f2b None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:34.716814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 748/1473] 10.222.0.82 () {54 vars in 967 bytes} [Wed Aug 24 03:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:34.721695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cd671-20d1-4b24-bb26-b25d452ce6fa None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:34.721982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cd671-20d1-4b24-bb26-b25d452ce6fa None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:34.722256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cd671-20d1-4b24-bb26-b25d452ce6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:34.799134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f66cd671-20d1-4b24-bb26-b25d452ce6fa None None] MFA Rules not processed for user `4ff3f53ddbb04eb8916b9ccfdb03dd80`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:34.800356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f66cd671-20d1-4b24-bb26-b25d452ce6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:34.821921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:34.877532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 726/1474] 10.222.0.82 () {58 vars in 1064 bytes} [Wed Aug 24 03:46:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:34.885303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a56f0-7aab-47e9-9edc-10f923115f2b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:34.886857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-145a56f0-7aab-47e9-9edc-10f923115f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:34.998284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a56f0-7aab-47e9-9edc-10f923115f2b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:35.075179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d8e29-7553-4895-8de4-c74ad3a24e4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ff3f53ddbb04eb8916b9ccfdb03dd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:35.076018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8e29-7553-4895-8de4-c74ad3a24e4f service glance] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:35.076199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8e29-7553-4895-8de4-c74ad3a24e4f service glance] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:35.076473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8e29-7553-4895-8de4-c74ad3a24e4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:35.078091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5c9d8e29-7553-4895-8de4-c74ad3a24e4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:35.133100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:35.188564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d8e29-7553-4895-8de4-c74ad3a24e4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:35.190029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d8e29-7553-4895-8de4-c74ad3a24e4f service glance] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:35.209688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 749/1475] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:37.226675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ac89c-e0af-4e2d-8c15-fff0313c387c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:37.227242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ac89c-e0af-4e2d-8c15-fff0313c387c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:37.227242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ac89c-e0af-4e2d-8c15-fff0313c387c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:37.228593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 727/1476] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:37.234624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474b1a1-be25-4b08-a7b5-00808e38a853 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:37.235070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474b1a1-be25-4b08-a7b5-00808e38a853 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:37.235177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474b1a1-be25-4b08-a7b5-00808e38a853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:37.301522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9474b1a1-be25-4b08-a7b5-00808e38a853 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:37.302697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9474b1a1-be25-4b08-a7b5-00808e38a853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:37.399549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 750/1477] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:37.686519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ac89c-e0af-4e2d-8c15-fff0313c387c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:37.688449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-743ac89c-e0af-4e2d-8c15-fff0313c387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:37.818513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ac89c-e0af-4e2d-8c15-fff0313c387c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:37.868316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:37.914386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941750e-41a2-4d08-bb90-f7f8a75416fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:37.915309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941750e-41a2-4d08-bb90-f7f8a75416fc admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:37.915590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941750e-41a2-4d08-bb90-f7f8a75416fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:37.915863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941750e-41a2-4d08-bb90-f7f8a75416fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:37.916505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941750e-41a2-4d08-bb90-f7f8a75416fc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:37.918272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WARNING py.warnings [None req-3941750e-41a2-4d08-bb90-f7f8a75416fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:46:37.918272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: warnings.warn(msg) Aug 24 03:46:37.918272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: Aug 24 03:46:37.920081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941750e-41a2-4d08-bb90-f7f8a75416fc admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:37.939672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 728/1478] 10.222.0.82 () {56 vars in 1108 bytes} [Wed Aug 24 03:46:37 2022] GET /identity/v3/projects => generated 2553 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:46:39.884465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8979b-eb2b-4c7c-bd03-a46322530531 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:39.884987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8979b-eb2b-4c7c-bd03-a46322530531 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:39.884987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8979b-eb2b-4c7c-bd03-a46322530531 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:39.886345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 751/1479] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:39.892718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a55121-59f9-4184-9fdd-a91f2cdc2d36 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:39.897061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a55121-59f9-4184-9fdd-a91f2cdc2d36 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:39.897061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a55121-59f9-4184-9fdd-a91f2cdc2d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:39.960137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-89a55121-59f9-4184-9fdd-a91f2cdc2d36 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:39.961362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-89a55121-59f9-4184-9fdd-a91f2cdc2d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:40.060197 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 729/1480] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:40.369481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8979b-eb2b-4c7c-bd03-a46322530531 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:40.371055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b8a8979b-eb2b-4c7c-bd03-a46322530531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:40.381807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:40.535927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8979b-eb2b-4c7c-bd03-a46322530531 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:40.631281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:40.671227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26779911-152b-41d1-a89b-f800e6c330c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:40.672016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26779911-152b-41d1-a89b-f800e6c330c8 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:40.672373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26779911-152b-41d1-a89b-f800e6c330c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:40.672652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26779911-152b-41d1-a89b-f800e6c330c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:40.675622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-26779911-152b-41d1-a89b-f800e6c330c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:40.772943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26779911-152b-41d1-a89b-f800e6c330c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:40.774709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26779911-152b-41d1-a89b-f800e6c330c8 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:40.795510 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 752/1481] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:42.910656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405445e-bc7c-4cb4-910c-31efe6012ead None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:42.911229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405445e-bc7c-4cb4-910c-31efe6012ead None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:42.911229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405445e-bc7c-4cb4-910c-31efe6012ead None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:42.912562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 730/1482] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:42.918207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29825ad9-1c6a-47f6-a742-28dc2ca77f71 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:42.918635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29825ad9-1c6a-47f6-a742-28dc2ca77f71 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:42.918795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29825ad9-1c6a-47f6-a742-28dc2ca77f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:42.987671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-29825ad9-1c6a-47f6-a742-28dc2ca77f71 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:42.988984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-29825ad9-1c6a-47f6-a742-28dc2ca77f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:43.039367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:43.091954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 753/1483] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:43.381569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405445e-bc7c-4cb4-910c-31efe6012ead None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:43.383200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b405445e-bc7c-4cb4-910c-31efe6012ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:43.407644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:43.488697 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405445e-bc7c-4cb4-910c-31efe6012ead None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:43.564920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b17cd-4d0f-491f-b2c4-1c662cb5afb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:43.565537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b17cd-4d0f-491f-b2c4-1c662cb5afb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:43.565823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b17cd-4d0f-491f-b2c4-1c662cb5afb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:43.566102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b17cd-4d0f-491f-b2c4-1c662cb5afb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:43.567775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fe9b17cd-4d0f-491f-b2c4-1c662cb5afb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:43.644127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:43.679704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b17cd-4d0f-491f-b2c4-1c662cb5afb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:43.681262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b17cd-4d0f-491f-b2c4-1c662cb5afb7 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:43.700974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 731/1484] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:45.697109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3334601-97bf-4b9d-9ebf-89be5458139f None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:45.697638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3334601-97bf-4b9d-9ebf-89be5458139f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:45.697638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3334601-97bf-4b9d-9ebf-89be5458139f None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:45.699008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 754/1485] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:45.705140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecd54-4a53-4ca4-975f-3d4fcf0fdc0c None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:45.705597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecd54-4a53-4ca4-975f-3d4fcf0fdc0c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:45.705707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecd54-4a53-4ca4-975f-3d4fcf0fdc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:45.777681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-952ecd54-4a53-4ca4-975f-3d4fcf0fdc0c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:45.779556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-952ecd54-4a53-4ca4-975f-3d4fcf0fdc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:45.877330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 732/1486] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:46.175128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3334601-97bf-4b9d-9ebf-89be5458139f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:46.176682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b3334601-97bf-4b9d-9ebf-89be5458139f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:46.282550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3334601-97bf-4b9d-9ebf-89be5458139f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:46.355918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:46.357830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e38b1b-865a-4f11-a357-c7b399cb6294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:46.358402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e38b1b-865a-4f11-a357-c7b399cb6294 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:46.359368 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e38b1b-865a-4f11-a357-c7b399cb6294 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:46.359653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e38b1b-865a-4f11-a357-c7b399cb6294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:46.361971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-60e38b1b-865a-4f11-a357-c7b399cb6294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:46.475771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e38b1b-865a-4f11-a357-c7b399cb6294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:46.477237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e38b1b-865a-4f11-a357-c7b399cb6294 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:46.496487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 755/1487] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:48.592272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48494be4-cb0c-4233-a106-e2b31c10a85a None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:48.592849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48494be4-cb0c-4233-a106-e2b31c10a85a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:48.592849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48494be4-cb0c-4233-a106-e2b31c10a85a None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:48.594208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 733/1488] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:48.599614 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5512c-624c-46b8-8233-cbb6d01bff12 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:48.600016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5512c-624c-46b8-8233-cbb6d01bff12 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:48.600172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5512c-624c-46b8-8233-cbb6d01bff12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:48.668187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b8a5512c-624c-46b8-8233-cbb6d01bff12 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:48.669448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b8a5512c-624c-46b8-8233-cbb6d01bff12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:48.723337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:48.768422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 756/1489] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:49.056740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48494be4-cb0c-4233-a106-e2b31c10a85a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:49.058287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-48494be4-cb0c-4233-a106-e2b31c10a85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:49.094467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:49.174499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48494be4-cb0c-4233-a106-e2b31c10a85a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:49.246746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d921d-ed59-4879-b883-e52cb26b38a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:49.247367 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d921d-ed59-4879-b883-e52cb26b38a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:49.248266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d921d-ed59-4879-b883-e52cb26b38a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:49.248961 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d921d-ed59-4879-b883-e52cb26b38a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:49.250647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e25d921d-ed59-4879-b883-e52cb26b38a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:49.332170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:49.361972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d921d-ed59-4879-b883-e52cb26b38a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:49.363457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d921d-ed59-4879-b883-e52cb26b38a4 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:49.382550 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 734/1490] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:51.694485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474ef15-84c3-4e7c-84b8-80d877336d74 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:51.695021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474ef15-84c3-4e7c-84b8-80d877336d74 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:51.695021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474ef15-84c3-4e7c-84b8-80d877336d74 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:51.696341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 757/1491] 10.222.0.82 () {54 vars in 927 bytes} [Wed Aug 24 03:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:46:51.702777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c554d9-d389-4b6b-a7f3-a656b1c63b5d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:51.703205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c554d9-d389-4b6b-a7f3-a656b1c63b5d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:51.703336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c554d9-d389-4b6b-a7f3-a656b1c63b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:51.768843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e5c554d9-d389-4b6b-a7f3-a656b1c63b5d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:51.770266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e5c554d9-d389-4b6b-a7f3-a656b1c63b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:51.873673 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 735/1492] 10.222.0.82 () {58 vars in 1024 bytes} [Wed Aug 24 03:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:52.181577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474ef15-84c3-4e7c-84b8-80d877336d74 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:52.183169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c474ef15-84c3-4e7c-84b8-80d877336d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:52.289928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474ef15-84c3-4e7c-84b8-80d877336d74 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:52.362391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099053cb-ea34-41d7-a6c5-80a2f4fa1f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:52.363027 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099053cb-ea34-41d7-a6c5-80a2f4fa1f33 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:52.363316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099053cb-ea34-41d7-a6c5-80a2f4fa1f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:52.363591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099053cb-ea34-41d7-a6c5-80a2f4fa1f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:52.365198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-099053cb-ea34-41d7-a6c5-80a2f4fa1f33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:52.374836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:52.476283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099053cb-ea34-41d7-a6c5-80a2f4fa1f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:52.477739 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099053cb-ea34-41d7-a6c5-80a2f4fa1f33 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:52.497665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 758/1493] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:47:01.871441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=82 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:47:02.495200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=50 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:23.575647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aa79a-d578-4236-b923-1799c79496c1 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:23.576206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aa79a-d578-4236-b923-1799c79496c1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:23.576206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aa79a-d578-4236-b923-1799c79496c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:23.652238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-fa8aa79a-d578-4236-b923-1799c79496c1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:23.653466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fa8aa79a-d578-4236-b923-1799c79496c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:23.749374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 736/1494] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:23.758019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fc0b6-8f51-4a05-92ca-7e27349dca46 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:23.758554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fc0b6-8f51-4a05-92ca-7e27349dca46 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:23.758665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fc0b6-8f51-4a05-92ca-7e27349dca46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:23.835159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-477fc0b6-8f51-4a05-92ca-7e27349dca46 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:23.836392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-477fc0b6-8f51-4a05-92ca-7e27349dca46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:23.932252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 759/1495] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:23.937036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8aa79a-d578-4236-b923-1799c79496c1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:23.938905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fa8aa79a-d578-4236-b923-1799c79496c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:23.950319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8aa79a-d578-4236-b923-1799c79496c1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:24.015563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:24.051552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:24.052231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:24.052515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:24.052787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:24.053432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:48:24.053736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:24.054796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WARNING py.warnings [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:48:24.054796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: warnings.warn(msg) Aug 24 03:48:24.054796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: Aug 24 03:48:24.055687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:24.068637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 737/1496] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:48:24.076981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fc0b6-8f51-4a05-92ca-7e27349dca46 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:24.086132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fc0b6-8f51-4a05-92ca-7e27349dca46 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:24.146301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:24.185470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:24.186088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:24.186396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:24.186706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:24.187389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:24.189873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WARNING py.warnings [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:48:24.189873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: warnings.warn(msg) Aug 24 03:48:24.189873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: Aug 24 03:48:24.191653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:24.373589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 760/1497] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:24 2022] POST /identity/v3/projects => generated 379 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:24.378295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:24.379957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:24.414591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efba07-d712-49d6-a267-83d9d1cb1ed9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:24.509615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:24.510491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:24.510950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:24.511378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:24.512349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:24.513891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WARNING py.warnings [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:48:24.513891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: warnings.warn(msg) Aug 24 03:48:24.513891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: Aug 24 03:48:24.515846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:24.658214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 738/1498] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:24 2022] POST /identity/v3/users => generated 290 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:24.662412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:24.664003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:24.728032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43af36-7ffe-4239-b85d-77c2d9dda25a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:24.822500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:24.833991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:24.834637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:24.834923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:24.835196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:24.835851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:24.836876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WARNING py.warnings [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:48:24.836876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: warnings.warn(msg) Aug 24 03:48:24.836876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: Aug 24 03:48:24.837767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:24.851821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 761/1499] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:24.856002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:24.857553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:24.869662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a1743-4b41-4690-a650-17d03efc2d41 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:24.894695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:24.966718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b41936-c49d-458b-a96c-b6e520903d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:24.967258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b41936-c49d-458b-a96c-b6e520903d43 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:24.967566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b41936-c49d-458b-a96c-b6e520903d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:24.967839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b41936-c49d-458b-a96c-b6e520903d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:24.968504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b41936-c49d-458b-a96c-b6e520903d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:24.969557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WARNING py.warnings [None req-d0b41936-c49d-458b-a96c-b6e520903d43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:48:24.969557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: warnings.warn(msg) Aug 24 03:48:24.969557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: Aug 24 03:48:24.970445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b41936-c49d-458b-a96c-b6e520903d43 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:24.983009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 739/1500] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:24.987657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:24.989182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:25.024298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208fc55-c446-48dc-b03c-c732d429ee32 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:25.120305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70380f6-edd3-4c39-ac2e-a270d8e51af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:25.120979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70380f6-edd3-4c39-ac2e-a270d8e51af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:25.121260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70380f6-edd3-4c39-ac2e-a270d8e51af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:25.121536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70380f6-edd3-4c39-ac2e-a270d8e51af5 admin admin] PATH_INFO: `/v3/projects/58135e820dbf4aa0b938bfae7251b18f/users/cf4a24f1466c4fa299ad4449c811f545/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:25.144247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70380f6-edd3-4c39-ac2e-a270d8e51af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58135e820dbf4aa0b938bfae7251b18f, user_id=cf4a24f1466c4fa299ad4449c811f545, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:25.145286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WARNING py.warnings [None req-c70380f6-edd3-4c39-ac2e-a270d8e51af5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:48:25.145286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: warnings.warn(msg) Aug 24 03:48:25.145286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: Aug 24 03:48:25.147260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70380f6-edd3-4c39-ac2e-a270d8e51af5 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:25.271722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 762/1501] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:24 2022] PUT /identity/v3/projects/58135e820dbf4aa0b938bfae7251b18f/users/cf4a24f1466c4fa299ad4449c811f545/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:25.284414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0029bce-d2a8-4894-bc9c-d5af850b6c79 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:25.284699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0029bce-d2a8-4894-bc9c-d5af850b6c79 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:25.284976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0029bce-d2a8-4894-bc9c-d5af850b6c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:25.340912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f0029bce-d2a8-4894-bc9c-d5af850b6c79 None None] MFA Rules not processed for user `cf4a24f1466c4fa299ad4449c811f545`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:25.342133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f0029bce-d2a8-4894-bc9c-d5af850b6c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:25.422813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 740/1502] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:48:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:25.435981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00711b3-c7b8-474a-92c1-8a7c2e667663 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:25.436270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00711b3-c7b8-474a-92c1-8a7c2e667663 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:25.436577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00711b3-c7b8-474a-92c1-8a7c2e667663 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:25.438068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 763/1503] 10.222.0.82 () {54 vars in 964 bytes} [Wed Aug 24 03:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:48:25.442823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05116d83-29d7-4b26-92c2-e4c8c495b05f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:25.443116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05116d83-29d7-4b26-92c2-e4c8c495b05f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:25.443391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05116d83-29d7-4b26-92c2-e4c8c495b05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:25.474737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:25.530404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-05116d83-29d7-4b26-92c2-e4c8c495b05f None None] MFA Rules not processed for user `c89caed732274718aeaa0d8ffeef571b`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:25.532246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-05116d83-29d7-4b26-92c2-e4c8c495b05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:25.611006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 741/1504] 10.222.0.82 () {58 vars in 1061 bytes} [Wed Aug 24 03:48:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:25.618815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00711b3-c7b8-474a-92c1-8a7c2e667663 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:25.620397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d00711b3-c7b8-474a-92c1-8a7c2e667663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:25.672993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:25.728325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00711b3-c7b8-474a-92c1-8a7c2e667663 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:25.803555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:25.804083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:25.804366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:25.804640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:25.806248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:25.907756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:25.909254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:25.913455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:25.942274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 764/1505] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:25.996338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268e37-9e41-4ed6-89cb-18890911099e None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:25.996771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268e37-9e41-4ed6-89cb-18890911099e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:25.996903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268e37-9e41-4ed6-89cb-18890911099e None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:25.998151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 742/1506] 10.222.0.82 () {52 vars in 830 bytes} [Wed Aug 24 03:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:48:26.002672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:26.003903 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:26.104435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595b34-1ddb-410a-80c9-286e57beacc2 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:26.198655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:26.199294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:26.199575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:26.199852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] PATH_INFO: `/v3/users/cf4a24f1466c4fa299ad4449c811f545` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:26.212811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf4a24f1466c4fa299ad4449c811f545)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:26.213855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WARNING py.warnings [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:48:26.213855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: warnings.warn(msg) Aug 24 03:48:26.213855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: Aug 24 03:48:26.215134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:26.225678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:26.560969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf4a24f1466c4fa299ad4449c811f545'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:26.561420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf4a24f1466c4fa299ad4449c811f545'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:26.698942 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf4a24f1466c4fa299ad4449c811f545'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:26.714812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf4a24f1466c4fa299ad4449c811f545'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:26.739708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf4a24f1466c4fa299ad4449c811f545'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:26.745457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 765/1507] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:26 2022] DELETE /identity/v3/users/cf4a24f1466c4fa299ad4449c811f545 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:26.750543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0264ba-fe89-4a46-a177-ed45af4ba4f9 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:26.750844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0264ba-fe89-4a46-a177-ed45af4ba4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:26.751120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0264ba-fe89-4a46-a177-ed45af4ba4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:26.812996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-3d0264ba-fe89-4a46-a177-ed45af4ba4f9 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:26.814324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3d0264ba-fe89-4a46-a177-ed45af4ba4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:26.897954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:26.911061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 743/1508] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:26.921050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:26.922415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:27.026037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa1901-8ac1-4dd5-a5c2-1160e09c3a55 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:27.085353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:27.106526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f33a9e-3ad1-4894-88fb-3252d56c9d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:27.143076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f33a9e-3ad1-4894-88fb-3252d56c9d62 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:27.143076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f33a9e-3ad1-4894-88fb-3252d56c9d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:27.143076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f33a9e-3ad1-4894-88fb-3252d56c9d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:27.143076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-70f33a9e-3ad1-4894-88fb-3252d56c9d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:27.230207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f33a9e-3ad1-4894-88fb-3252d56c9d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:27.231786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f33a9e-3ad1-4894-88fb-3252d56c9d62 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:27.251491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 766/1509] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:48:28.009726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0264ba-fe89-4a46-a177-ed45af4ba4f9 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:28.011078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3d0264ba-fe89-4a46-a177-ed45af4ba4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:28.025435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0264ba-fe89-4a46-a177-ed45af4ba4f9 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:28.128808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:28.129720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:28.130166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:28.130653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] PATH_INFO: `/v3/projects/58135e820dbf4aa0b938bfae7251b18f` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:28.145521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=58135e820dbf4aa0b938bfae7251b18f)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:28.146676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WARNING py.warnings [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:48:28.146676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: warnings.warn(msg) Aug 24 03:48:28.146676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: Aug 24 03:48:28.148046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:28.289845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58135e820dbf4aa0b938bfae7251b18f', 'request_id': 'req-f6b94576-0d49-4193-b977-819aa5df0cfe'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:28.405015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58135e820dbf4aa0b938bfae7251b18f', 'request_id': 'req-f6b94576-0d49-4193-b977-819aa5df0cfe'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:28.427995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58135e820dbf4aa0b938bfae7251b18f', 'request_id': 'req-f6b94576-0d49-4193-b977-819aa5df0cfe'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:28.452455 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] The token cache is being invalidate because project 58135e820dbf4aa0b938bfae7251b18f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:28.452806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:28.453310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f6b94576-0d49-4193-b977-819aa5df0cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:28.454826 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 744/1510] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:28 2022] DELETE /identity/v3/projects/58135e820dbf4aa0b938bfae7251b18f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:37.622693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=62 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:38.450108 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:02.234068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfab89-a823-48cd-b4a1-8fed98f24309 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:02.235018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfab89-a823-48cd-b4a1-8fed98f24309 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:02.235018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfab89-a823-48cd-b4a1-8fed98f24309 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:02.236039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:50:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:50:02.242288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a42ed9-9d50-4449-858c-14621707be36 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:02.242973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a42ed9-9d50-4449-858c-14621707be36 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:02.242973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a42ed9-9d50-4449-858c-14621707be36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:02.347642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e7a42ed9-9d50-4449-858c-14621707be36 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:02.350054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e7a42ed9-9d50-4449-858c-14621707be36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:02.472707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:02.629100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfab89-a823-48cd-b4a1-8fed98f24309 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:02.632455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7bbfab89-a823-48cd-b4a1-8fed98f24309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:02.756849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfab89-a823-48cd-b4a1-8fed98f24309 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:02.762497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b557a5-ddf5-462b-898b-17bdbddf37f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:02.763140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b557a5-ddf5-462b-898b-17bdbddf37f8 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:02.763426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b557a5-ddf5-462b-898b-17bdbddf37f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:02.763723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b557a5-ddf5-462b-898b-17bdbddf37f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:02.772955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b557a5-ddf5-462b-898b-17bdbddf37f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:02.774698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b557a5-ddf5-462b-898b-17bdbddf37f8 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:02.784363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 768/1513] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:03.140102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d622ef1-2dcd-4a50-a0b8-1a1c261937fd None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:03.140476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d622ef1-2dcd-4a50-a0b8-1a1c261937fd None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:03.140776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d622ef1-2dcd-4a50-a0b8-1a1c261937fd None None] PATH_INFO: `/v3` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:03.142132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 746/1514] 10.222.0.83 () {54 vars in 925 bytes} [Wed Aug 24 03:50:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:50:03.148843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a23ac-3c60-4d8b-9f9e-4bb22d735a54 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:03.149132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a23ac-3c60-4d8b-9f9e-4bb22d735a54 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:03.149410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a23ac-3c60-4d8b-9f9e-4bb22d735a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:03.217362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5c1a23ac-3c60-4d8b-9f9e-4bb22d735a54 None None] MFA Rules not processed for user `cff23ae056ff452588cd0b6520e96962`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:03.218619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5c1a23ac-3c60-4d8b-9f9e-4bb22d735a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:03.291703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:03.296836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 769/1515] 10.222.0.83 () {58 vars in 1015 bytes} [Wed Aug 24 03:50:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:03.640520 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14913a3b-a2e9-4c10-a733-f0b4b26aece8 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:03.641097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14913a3b-a2e9-4c10-a733-f0b4b26aece8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:03.641097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14913a3b-a2e9-4c10-a733-f0b4b26aece8 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:03.642453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 747/1516] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:50:03.648166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55b129-4043-45c4-bf6f-cf71dbe42021 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:03.648461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55b129-4043-45c4-bf6f-cf71dbe42021 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:03.648737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55b129-4043-45c4-bf6f-cf71dbe42021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:03.669001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-7e55b129-4043-45c4-bf6f-cf71dbe42021 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:03.670225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7e55b129-4043-45c4-bf6f-cf71dbe42021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:03.679462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 770/1517] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:03.832121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14913a3b-a2e9-4c10-a733-f0b4b26aece8 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:03.847819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14913a3b-a2e9-4c10-a733-f0b4b26aece8 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:03.853791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc590e-de8b-4258-ae18-0df93e2f38b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:03.854394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc590e-de8b-4258-ae18-0df93e2f38b0 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:03.854723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc590e-de8b-4258-ae18-0df93e2f38b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:03.855000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc590e-de8b-4258-ae18-0df93e2f38b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:03.863137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc590e-de8b-4258-ae18-0df93e2f38b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:03.864840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc590e-de8b-4258-ae18-0df93e2f38b0 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:03.874430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 748/1518] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:50:12.580833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f68cfc-4c10-453b-aac6-ee0c4c58ce5e None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:12.581403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f68cfc-4c10-453b-aac6-ee0c4c58ce5e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:12.581403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f68cfc-4c10-453b-aac6-ee0c4c58ce5e None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:12.583208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 771/1519] 10.222.0.82 () {54 vars in 961 bytes} [Wed Aug 24 03:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:50:12.588743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd478490-f227-4f1d-8222-b20d2343ecad None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:12.589196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd478490-f227-4f1d-8222-b20d2343ecad None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:12.589310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd478490-f227-4f1d-8222-b20d2343ecad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:12.609193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-dd478490-f227-4f1d-8222-b20d2343ecad None None] MFA Rules not processed for user `cff23ae056ff452588cd0b6520e96962`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:12.610384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dd478490-f227-4f1d-8222-b20d2343ecad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:12.619001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 749/1520] 10.222.0.82 () {58 vars in 1058 bytes} [Wed Aug 24 03:50:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:12.626710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f68cfc-4c10-453b-aac6-ee0c4c58ce5e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:12.640141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f68cfc-4c10-453b-aac6-ee0c4c58ce5e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:12.645089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47524603-b8b4-42af-a63c-0e976f80c3d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:12.646490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47524603-b8b4-42af-a63c-0e976f80c3d7 service placement] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:12.646644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47524603-b8b4-42af-a63c-0e976f80c3d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:12.646836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47524603-b8b4-42af-a63c-0e976f80c3d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:12.655070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47524603-b8b4-42af-a63c-0e976f80c3d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:12.656679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47524603-b8b4-42af-a63c-0e976f80c3d7 service placement] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:12.666927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 772/1521] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:15.119189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f620d6-f29c-4579-b8c4-35aa6aae79c5 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:15.119758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f620d6-f29c-4579-b8c4-35aa6aae79c5 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:15.119758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f620d6-f29c-4579-b8c4-35aa6aae79c5 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:15.121087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:50:15.125922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13b247-9a28-458a-bd3d-f5833978da0b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:15.126365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13b247-9a28-458a-bd3d-f5833978da0b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:15.126519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13b247-9a28-458a-bd3d-f5833978da0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:15.146643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-cb13b247-9a28-458a-bd3d-f5833978da0b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:15.147842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-cb13b247-9a28-458a-bd3d-f5833978da0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:15.155676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:15.328054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f620d6-f29c-4579-b8c4-35aa6aae79c5 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:15.342644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f620d6-f29c-4579-b8c4-35aa6aae79c5 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:15.348013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cac3b0-c80a-483d-8e98-ce8d9ec7050d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:15.349240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cac3b0-c80a-483d-8e98-ce8d9ec7050d service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:15.349596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cac3b0-c80a-483d-8e98-ce8d9ec7050d service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:15.349893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cac3b0-c80a-483d-8e98-ce8d9ec7050d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:15.358017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cac3b0-c80a-483d-8e98-ce8d9ec7050d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:15.359632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cac3b0-c80a-483d-8e98-ce8d9ec7050d service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:15.369696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 751/1524] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:16.194113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064a5e8-eb4e-4f1a-aa92-3ebe56fc7e58 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:16.194695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064a5e8-eb4e-4f1a-aa92-3ebe56fc7e58 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:16.194695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064a5e8-eb4e-4f1a-aa92-3ebe56fc7e58 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:16.196102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:50:16.201160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882e5de-fbff-47b4-a4f0-dadf3f1eba78 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:16.201584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882e5de-fbff-47b4-a4f0-dadf3f1eba78 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:16.201718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882e5de-fbff-47b4-a4f0-dadf3f1eba78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:16.221919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-3882e5de-fbff-47b4-a4f0-dadf3f1eba78 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:16.223154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3882e5de-fbff-47b4-a4f0-dadf3f1eba78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:16.231715 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:16.386849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064a5e8-eb4e-4f1a-aa92-3ebe56fc7e58 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:16.401966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064a5e8-eb4e-4f1a-aa92-3ebe56fc7e58 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:16.407396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e73f6b-fcd5-48be-8135-d7ea04efdff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:16.407993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e73f6b-fcd5-48be-8135-d7ea04efdff5 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:16.408275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e73f6b-fcd5-48be-8135-d7ea04efdff5 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:16.408550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e73f6b-fcd5-48be-8135-d7ea04efdff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:16.417240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e73f6b-fcd5-48be-8135-d7ea04efdff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:16.418871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e73f6b-fcd5-48be-8135-d7ea04efdff5 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:16.428838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 775/1527] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:20.580889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea51331-8ee3-4536-8f90-d802d8ceb69c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:20.581497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea51331-8ee3-4536-8f90-d802d8ceb69c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:20.581497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea51331-8ee3-4536-8f90-d802d8ceb69c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:20.582825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:50:20.588256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b26ba-3fa7-4010-ac91-058bc01cfe9b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:20.588679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b26ba-3fa7-4010-ac91-058bc01cfe9b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:20.588803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b26ba-3fa7-4010-ac91-058bc01cfe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:20.608849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a69b26ba-3fa7-4010-ac91-058bc01cfe9b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:20.610060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a69b26ba-3fa7-4010-ac91-058bc01cfe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:20.618709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:20.905504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea51331-8ee3-4536-8f90-d802d8ceb69c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:20.907246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8ea51331-8ee3-4536-8f90-d802d8ceb69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:20.940806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:20.987583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea51331-8ee3-4536-8f90-d802d8ceb69c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:20.995072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dd120-6024-4a98-a34f-967ac995cf35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:20.995698 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dd120-6024-4a98-a34f-967ac995cf35 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:20.995986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dd120-6024-4a98-a34f-967ac995cf35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:20.996271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dd120-6024-4a98-a34f-967ac995cf35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:21.008212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dd120-6024-4a98-a34f-967ac995cf35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:21.009688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dd120-6024-4a98-a34f-967ac995cf35 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:21.020063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 754/1530] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:22.927771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217a64e-088f-4fb1-a806-7787b1f05820 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:22.928815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217a64e-088f-4fb1-a806-7787b1f05820 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:22.928940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217a64e-088f-4fb1-a806-7787b1f05820 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:22.930268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:50:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:50:22.935819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb1def-107e-4869-abb4-32ba5ffa75f8 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:22.936231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb1def-107e-4869-abb4-32ba5ffa75f8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:22.936379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb1def-107e-4869-abb4-32ba5ffa75f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:22.956549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b3eb1def-107e-4869-abb4-32ba5ffa75f8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:22.957780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b3eb1def-107e-4869-abb4-32ba5ffa75f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:22.966856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:23.263109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217a64e-088f-4fb1-a806-7787b1f05820 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:23.275009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217a64e-088f-4fb1-a806-7787b1f05820 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:23.278520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d37953-731d-4359-8f73-65bd9712e314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:23.279127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d37953-731d-4359-8f73-65bd9712e314 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.279414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d37953-731d-4359-8f73-65bd9712e314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.279694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d37953-731d-4359-8f73-65bd9712e314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.289539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d37953-731d-4359-8f73-65bd9712e314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:23.291029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d37953-731d-4359-8f73-65bd9712e314 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:23.300440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 778/1533] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:26.776656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4284778-98a1-4c63-835e-abce2327d93c None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:26.777202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4284778-98a1-4c63-835e-abce2327d93c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:26.777202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4284778-98a1-4c63-835e-abce2327d93c None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:26.778577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:50:26.783967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c0102-a1a6-41a6-a569-8a0693736dfb None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:26.784420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c0102-a1a6-41a6-a569-8a0693736dfb None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:26.784530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c0102-a1a6-41a6-a569-8a0693736dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:26.805269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ec5c0102-a1a6-41a6-a569-8a0693736dfb None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:26.806491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ec5c0102-a1a6-41a6-a569-8a0693736dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:26.814440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:50:27.374326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4284778-98a1-4c63-835e-abce2327d93c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:27.383192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a4284778-98a1-4c63-835e-abce2327d93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:27.457922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4284778-98a1-4c63-835e-abce2327d93c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:27.462004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79547c45-5eac-4ac5-9c16-aa09294548e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:27.462640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79547c45-5eac-4ac5-9c16-aa09294548e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:27.462929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79547c45-5eac-4ac5-9c16-aa09294548e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:27.463207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79547c45-5eac-4ac5-9c16-aa09294548e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:27.472140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79547c45-5eac-4ac5-9c16-aa09294548e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:27.473594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79547c45-5eac-4ac5-9c16-aa09294548e8 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:27.483583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 757/1536] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:29.419153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211c98b-7263-467e-bb8b-39103288485d None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:29.419722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211c98b-7263-467e-bb8b-39103288485d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:29.419722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211c98b-7263-467e-bb8b-39103288485d None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:29.421057 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:50:29.426337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08741718-af75-4e8c-9b16-073c8f57f749 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:29.426772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08741718-af75-4e8c-9b16-073c8f57f749 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:29.426943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08741718-af75-4e8c-9b16-073c8f57f749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:29.448004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-08741718-af75-4e8c-9b16-073c8f57f749 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:29.449195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-08741718-af75-4e8c-9b16-073c8f57f749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:29.456938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:30.031103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211c98b-7263-467e-bb8b-39103288485d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:30.032794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3211c98b-7263-467e-bb8b-39103288485d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:30.046255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211c98b-7263-467e-bb8b-39103288485d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:30.050067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee5f66-550a-4652-9d29-9aaa7e7b6b48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:30.050707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee5f66-550a-4652-9d29-9aaa7e7b6b48 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:30.050993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee5f66-550a-4652-9d29-9aaa7e7b6b48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:30.051271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee5f66-550a-4652-9d29-9aaa7e7b6b48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:30.061195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee5f66-550a-4652-9d29-9aaa7e7b6b48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:30.062663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee5f66-550a-4652-9d29-9aaa7e7b6b48 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:30.072904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 781/1539] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:39.442790 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=63 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:40.067865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=37 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:55.647563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33180e-6b08-4c36-9b05-bebba66b1f47 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:55.649001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33180e-6b08-4c36-9b05-bebba66b1f47 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:55.649001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33180e-6b08-4c36-9b05-bebba66b1f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:55.673830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-bd33180e-6b08-4c36-9b05-bebba66b1f47 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:55.675573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bd33180e-6b08-4c36-9b05-bebba66b1f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:55.689964 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 759/1540] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:51:55.699577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad8e6a-a8f1-44fc-a433-aeb925bf4ec1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:55.700888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad8e6a-a8f1-44fc-a433-aeb925bf4ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:55.700888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad8e6a-a8f1-44fc-a433-aeb925bf4ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:55.731899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9fad8e6a-a8f1-44fc-a433-aeb925bf4ec1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:55.733139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9fad8e6a-a8f1-44fc-a433-aeb925bf4ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:55.746154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 782/1541] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:55.751337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33180e-6b08-4c36-9b05-bebba66b1f47 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:55.762590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33180e-6b08-4c36-9b05-bebba66b1f47 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:55.771573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ee8ff-6cdb-455e-a9b2-bd995ed09c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:55.772225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ee8ff-6cdb-455e-a9b2-bd995ed09c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:55.772509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ee8ff-6cdb-455e-a9b2-bd995ed09c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:55.772784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ee8ff-6cdb-455e-a9b2-bd995ed09c3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:55.773504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ee8ff-6cdb-455e-a9b2-bd995ed09c3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:51:55.773751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ee8ff-6cdb-455e-a9b2-bd995ed09c3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:55.775513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ee8ff-6cdb-455e-a9b2-bd995ed09c3b admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:55.806845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 760/1542] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:51:55.816163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad8e6a-a8f1-44fc-a433-aeb925bf4ec1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:55.831617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad8e6a-a8f1-44fc-a433-aeb925bf4ec1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:55.839677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9034352-7f4e-41a7-bf53-9e5370013d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:55.840325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9034352-7f4e-41a7-bf53-9e5370013d9a admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:55.840610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9034352-7f4e-41a7-bf53-9e5370013d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:55.840887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9034352-7f4e-41a7-bf53-9e5370013d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:55.841611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9034352-7f4e-41a7-bf53-9e5370013d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:55.843596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9034352-7f4e-41a7-bf53-9e5370013d9a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:55.959093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed5cdc-1903-4654-bedb-befe76c06696 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:55.959696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed5cdc-1903-4654-bedb-befe76c06696 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:55.961094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed5cdc-1903-4654-bedb-befe76c06696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:55.990265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 783/1543] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:55 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:55.990708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef059bb-c36c-45bb-93b6-cf009da72749 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:55.991000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef059bb-c36c-45bb-93b6-cf009da72749 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:55.991335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef059bb-c36c-45bb-93b6-cf009da72749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:55.993725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c4ed5cdc-1903-4654-bedb-befe76c06696 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:55.995775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c4ed5cdc-1903-4654-bedb-befe76c06696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.043582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-cef059bb-c36c-45bb-93b6-cf009da72749 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.044830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-cef059bb-c36c-45bb-93b6-cf009da72749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.096391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 761/1544] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:51:56.097323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed5cdc-1903-4654-bedb-befe76c06696 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.108580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed5cdc-1903-4654-bedb-befe76c06696 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.115083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7cc68-5097-40ba-aee4-a5b530516cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.115924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7cc68-5097-40ba-aee4-a5b530516cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.116140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7cc68-5097-40ba-aee4-a5b530516cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.116420 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7cc68-5097-40ba-aee4-a5b530516cdb admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.117064 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7cc68-5097-40ba-aee4-a5b530516cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.119051 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7cc68-5097-40ba-aee4-a5b530516cdb admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.139222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 784/1545] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.140565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bacc6b-04e5-4b26-b000-3c27f9582fbc None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.140863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bacc6b-04e5-4b26-b000-3c27f9582fbc None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.141129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bacc6b-04e5-4b26-b000-3c27f9582fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.161502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-64bacc6b-04e5-4b26-b000-3c27f9582fbc None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.162808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-64bacc6b-04e5-4b26-b000-3c27f9582fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.171163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 785/1546] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.182741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88162409-450c-4669-9046-7d12c9f2671e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.182741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88162409-450c-4669-9046-7d12c9f2671e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.182741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88162409-450c-4669-9046-7d12c9f2671e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.193981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-88162409-450c-4669-9046-7d12c9f2671e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.195250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-88162409-450c-4669-9046-7d12c9f2671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.202359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 762/1547] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:56.203766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06d0a6-185c-4031-8a0f-6a76ecc14f85 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.207846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06d0a6-185c-4031-8a0f-6a76ecc14f85 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.207846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06d0a6-185c-4031-8a0f-6a76ecc14f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.208157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 786/1548] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.208157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdbbe1-4286-4908-9881-e36e64d040a8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.208157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdbbe1-4286-4908-9881-e36e64d040a8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.208157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdbbe1-4286-4908-9881-e36e64d040a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.229723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-8f06d0a6-185c-4031-8a0f-6a76ecc14f85 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.230863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8f06d0a6-185c-4031-8a0f-6a76ecc14f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.232103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9cbdbbe1-4286-4908-9881-e36e64d040a8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.233275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9cbdbbe1-4286-4908-9881-e36e64d040a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.239630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 763/1549] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.240845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551280b-df4e-4a78-b2bc-3b0d91b3aba3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.241130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551280b-df4e-4a78-b2bc-3b0d91b3aba3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.241403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551280b-df4e-4a78-b2bc-3b0d91b3aba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.246667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 787/1550] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.248133 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c427d-7659-4ed4-9e21-56a8012425c7 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.248425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c427d-7659-4ed4-9e21-56a8012425c7 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.248700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c427d-7659-4ed4-9e21-56a8012425c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.264251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b551280b-df4e-4a78-b2bc-3b0d91b3aba3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.265941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b551280b-df4e-4a78-b2bc-3b0d91b3aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.268265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-984c427d-7659-4ed4-9e21-56a8012425c7 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.269438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-984c427d-7659-4ed4-9e21-56a8012425c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.275218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 764/1551] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.276492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bafa42-6e27-46db-b972-fada23bf33ea None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.276741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bafa42-6e27-46db-b972-fada23bf33ea None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.277055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bafa42-6e27-46db-b972-fada23bf33ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.279966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 788/1552] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.281633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab511afa-4471-4327-8039-ddf4e0afca95 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.282015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab511afa-4471-4327-8039-ddf4e0afca95 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.282308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab511afa-4471-4327-8039-ddf4e0afca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.298033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-51bafa42-6e27-46db-b972-fada23bf33ea None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.299253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-51bafa42-6e27-46db-b972-fada23bf33ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.307006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ab511afa-4471-4327-8039-ddf4e0afca95 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.307159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 765/1553] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.307920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bafa42-6e27-46db-b972-fada23bf33ea None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.308268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ab511afa-4471-4327-8039-ddf4e0afca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.318915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 789/1554] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.320297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b59fc6-2d06-403d-84e7-f94f13225c2f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.320589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b59fc6-2d06-403d-84e7-f94f13225c2f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.320865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b59fc6-2d06-403d-84e7-f94f13225c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.326042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bafa42-6e27-46db-b972-fada23bf33ea None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.332483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67148b-3de1-42f9-a9a9-fed4e0c086ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.333109 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67148b-3de1-42f9-a9a9-fed4e0c086ad admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.333397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67148b-3de1-42f9-a9a9-fed4e0c086ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.333671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67148b-3de1-42f9-a9a9-fed4e0c086ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.334330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67148b-3de1-42f9-a9a9-fed4e0c086ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.336454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67148b-3de1-42f9-a9a9-fed4e0c086ad admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.341158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-14b59fc6-2d06-403d-84e7-f94f13225c2f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.342660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-14b59fc6-2d06-403d-84e7-f94f13225c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.347737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 766/1555] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:56.348916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a369cd-e31d-4dea-8e9c-613b86541d1b None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.349253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a369cd-e31d-4dea-8e9c-613b86541d1b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.349544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a369cd-e31d-4dea-8e9c-613b86541d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.351878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 790/1556] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.352739 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b59fc6-2d06-403d-84e7-f94f13225c2f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.364065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b59fc6-2d06-403d-84e7-f94f13225c2f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.368751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.369387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.369668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.369960 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.370652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:51:56.370961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.371994 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WARNING py.warnings [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:51:56.371994 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: warnings.warn(msg) Aug 24 03:51:56.371994 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: Aug 24 03:51:56.372882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.373989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c0a369cd-e31d-4dea-8e9c-613b86541d1b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.375859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c0a369cd-e31d-4dea-8e9c-613b86541d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.384416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 767/1557] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.393176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 791/1558] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:51:56.393353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6a581-3aca-4baf-a50f-9efe6c69d7d5 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.393353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6a581-3aca-4baf-a50f-9efe6c69d7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.393353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6a581-3aca-4baf-a50f-9efe6c69d7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.393645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.403327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f0dab-0b2b-4f4c-86a4-85ad99212887 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.406260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2fd6a581-3aca-4baf-a50f-9efe6c69d7d5 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.407529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2fd6a581-3aca-4baf-a50f-9efe6c69d7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.408738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacc176-7d79-4c6c-9042-102fd22e0c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.409359 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacc176-7d79-4c6c-9042-102fd22e0c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.409648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacc176-7d79-4c6c-9042-102fd22e0c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.409928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacc176-7d79-4c6c-9042-102fd22e0c90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.410629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacc176-7d79-4c6c-9042-102fd22e0c90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:51:56.410940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacc176-7d79-4c6c-9042-102fd22e0c90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.412555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacc176-7d79-4c6c-9042-102fd22e0c90 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.417204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 768/1559] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.417355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ee28a-005d-48d1-8185-de8d66b3eb4d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.426750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ee28a-005d-48d1-8185-de8d66b3eb4d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.426750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ee28a-005d-48d1-8185-de8d66b3eb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.426986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 792/1560] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:51:56.427624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacc176-7d79-4c6c-9042-102fd22e0c90 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.439070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacc176-7d79-4c6c-9042-102fd22e0c90 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.443694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37d68b-4b8c-4bbd-82f3-74eb5afd54b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.444331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37d68b-4b8c-4bbd-82f3-74eb5afd54b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.444616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37d68b-4b8c-4bbd-82f3-74eb5afd54b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.444893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37d68b-4b8c-4bbd-82f3-74eb5afd54b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.445541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37d68b-4b8c-4bbd-82f3-74eb5afd54b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:51:56.445851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37d68b-4b8c-4bbd-82f3-74eb5afd54b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.447053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-bb7ee28a-005d-48d1-8185-de8d66b3eb4d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:56.447493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37d68b-4b8c-4bbd-82f3-74eb5afd54b6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.448275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bb7ee28a-005d-48d1-8185-de8d66b3eb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:56.458959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 769/1561] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:56.459776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 793/1562] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:51:56.460479 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37d68b-4b8c-4bbd-82f3-74eb5afd54b6 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.466879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ee28a-005d-48d1-8185-de8d66b3eb4d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.471757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37d68b-4b8c-4bbd-82f3-74eb5afd54b6 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.476512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04a986-0a0b-4e40-b0b9-7253b2eabbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.477113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04a986-0a0b-4e40-b0b9-7253b2eabbcc admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.477399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04a986-0a0b-4e40-b0b9-7253b2eabbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.477527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ee28a-005d-48d1-8185-de8d66b3eb4d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.477679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04a986-0a0b-4e40-b0b9-7253b2eabbcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.478360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04a986-0a0b-4e40-b0b9-7253b2eabbcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:51:56.478703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04a986-0a0b-4e40-b0b9-7253b2eabbcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.480460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04a986-0a0b-4e40-b0b9-7253b2eabbcc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.482190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dff232-7743-472c-9862-d455daaf174d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.482828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dff232-7743-472c-9862-d455daaf174d admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.483172 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dff232-7743-472c-9862-d455daaf174d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.483478 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dff232-7743-472c-9862-d455daaf174d admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.484176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dff232-7743-472c-9862-d455daaf174d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.486546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dff232-7743-472c-9862-d455daaf174d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.493743 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 794/1563] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:51:56.494603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04a986-0a0b-4e40-b0b9-7253b2eabbcc admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.505747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 770/1564] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:56.505747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dff232-7743-472c-9862-d455daaf174d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.510209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04a986-0a0b-4e40-b0b9-7253b2eabbcc admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.513794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dff232-7743-472c-9862-d455daaf174d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.514961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a5efd-5577-4a2d-8510-96f424bc3f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.515664 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a5efd-5577-4a2d-8510-96f424bc3f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.515944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a5efd-5577-4a2d-8510-96f424bc3f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.516216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a5efd-5577-4a2d-8510-96f424bc3f76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.516870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a5efd-5577-4a2d-8510-96f424bc3f76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:51:56.517216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a5efd-5577-4a2d-8510-96f424bc3f76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.518951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a5efd-5577-4a2d-8510-96f424bc3f76 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.519814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27964bfa-773e-4cb0-8ce7-180ea5ce554f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.520501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27964bfa-773e-4cb0-8ce7-180ea5ce554f admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.520852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27964bfa-773e-4cb0-8ce7-180ea5ce554f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.521197 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27964bfa-773e-4cb0-8ce7-180ea5ce554f admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.522027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27964bfa-773e-4cb0-8ce7-180ea5ce554f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.523349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WARNING py.warnings [None req-27964bfa-773e-4cb0-8ce7-180ea5ce554f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:51:56.523349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: warnings.warn(msg) Aug 24 03:51:56.523349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: Aug 24 03:51:56.524560 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27964bfa-773e-4cb0-8ce7-180ea5ce554f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.531755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 795/1565] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:51:56.535499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a5efd-5577-4a2d-8510-96f424bc3f76 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.545788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a5efd-5577-4a2d-8510-96f424bc3f76 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.551733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cc591-2694-4b30-963d-d6f4d4c27314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.552366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cc591-2694-4b30-963d-d6f4d4c27314 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.552648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cc591-2694-4b30-963d-d6f4d4c27314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.552923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cc591-2694-4b30-963d-d6f4d4c27314 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.553570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cc591-2694-4b30-963d-d6f4d4c27314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:51:56.553879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cc591-2694-4b30-963d-d6f4d4c27314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.555533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40cc591-2694-4b30-963d-d6f4d4c27314 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.567667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 796/1566] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:51:56.568482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cc591-2694-4b30-963d-d6f4d4c27314 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.575604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:56.579206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cc591-2694-4b30-963d-d6f4d4c27314 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.589946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c5d2-2cad-4069-a334-282f1486ef0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.589946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c5d2-2cad-4069-a334-282f1486ef0b admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.589946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c5d2-2cad-4069-a334-282f1486ef0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.589946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c5d2-2cad-4069-a334-282f1486ef0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.590444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45c5d2-2cad-4069-a334-282f1486ef0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.590444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45c5d2-2cad-4069-a334-282f1486ef0b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.649884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 771/1567] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:56.650771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27964bfa-773e-4cb0-8ce7-180ea5ce554f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.694844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:56.701049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 797/1568] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:56.705352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c5d2-2cad-4069-a334-282f1486ef0b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.714092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27964bfa-773e-4cb0-8ce7-180ea5ce554f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.770253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c5d2-2cad-4069-a334-282f1486ef0b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.772870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7051f52-bc4a-4fcd-8d4a-f624d6bbf218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.773489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7051f52-bc4a-4fcd-8d4a-f624d6bbf218 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.773772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7051f52-bc4a-4fcd-8d4a-f624d6bbf218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.774047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7051f52-bc4a-4fcd-8d4a-f624d6bbf218 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.774726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7051f52-bc4a-4fcd-8d4a-f624d6bbf218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:51:56.775032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7051f52-bc4a-4fcd-8d4a-f624d6bbf218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.775148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7efca1-4897-42b2-90d9-54bfdfeca6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.775691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7efca1-4897-42b2-90d9-54bfdfeca6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.775975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7efca1-4897-42b2-90d9-54bfdfeca6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.776248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7efca1-4897-42b2-90d9-54bfdfeca6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.776633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7051f52-bc4a-4fcd-8d4a-f624d6bbf218 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.776876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7efca1-4897-42b2-90d9-54bfdfeca6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.778861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7efca1-4897-42b2-90d9-54bfdfeca6d2 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.795968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 772/1569] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:51:56.795968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7051f52-bc4a-4fcd-8d4a-f624d6bbf218 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.800937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7051f52-bc4a-4fcd-8d4a-f624d6bbf218 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.805837 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aade9e-643b-471d-930a-59e21b546fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.806448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aade9e-643b-471d-930a-59e21b546fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.806762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aade9e-643b-471d-930a-59e21b546fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.807039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aade9e-643b-471d-930a-59e21b546fe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.807703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aade9e-643b-471d-930a-59e21b546fe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.809655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aade9e-643b-471d-930a-59e21b546fe1 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:56.872274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 798/1570] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/projects => generated 389 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:56.873123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7efca1-4897-42b2-90d9-54bfdfeca6d2 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.904282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 773/1571] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/projects => generated 391 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:56.904282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aade9e-643b-471d-930a-59e21b546fe1 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:56.952109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7efca1-4897-42b2-90d9-54bfdfeca6d2 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.969525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aade9e-643b-471d-930a-59e21b546fe1 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:56.974422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc837b5-4369-49a9-a56e-ee55b645c8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:56.975090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc837b5-4369-49a9-a56e-ee55b645c8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:56.975371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc837b5-4369-49a9-a56e-ee55b645c8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:56.975663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc837b5-4369-49a9-a56e-ee55b645c8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:56.976282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc837b5-4369-49a9-a56e-ee55b645c8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:56.978282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc837b5-4369-49a9-a56e-ee55b645c8b8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.013234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385fda5-ad4b-4eee-beaa-1fa81bfdc63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.013915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385fda5-ad4b-4eee-beaa-1fa81bfdc63c admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.014204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385fda5-ad4b-4eee-beaa-1fa81bfdc63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.014486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385fda5-ad4b-4eee-beaa-1fa81bfdc63c admin admin] PATH_INFO: `/v3/projects/649f222d62844debb70df5ed5e8d44ad/users/a466f40eba2a4975bd13614474ff527c/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.030781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385fda5-ad4b-4eee-beaa-1fa81bfdc63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=649f222d62844debb70df5ed5e8d44ad, user_id=a466f40eba2a4975bd13614474ff527c, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.034118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385fda5-ad4b-4eee-beaa-1fa81bfdc63c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.072101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 774/1572] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:56 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:57.072528 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc837b5-4369-49a9-a56e-ee55b645c8b8 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.119318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:57.136371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc837b5-4369-49a9-a56e-ee55b645c8b8 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.141668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422df5d-93cd-450d-9229-874f1484b344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.142281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422df5d-93cd-450d-9229-874f1484b344 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.142594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422df5d-93cd-450d-9229-874f1484b344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.142875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422df5d-93cd-450d-9229-874f1484b344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.143546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422df5d-93cd-450d-9229-874f1484b344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.145464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422df5d-93cd-450d-9229-874f1484b344 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.197350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:57.204454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 799/1573] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:56 2022] PUT /identity/v3/projects/649f222d62844debb70df5ed5e8d44ad/users/a466f40eba2a4975bd13614474ff527c/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:57.205251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385fda5-ad4b-4eee-beaa-1fa81bfdc63c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.265654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385fda5-ad4b-4eee-beaa-1fa81bfdc63c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.270689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacb3b7-1fc6-4d3a-b98a-98619de367b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.271314 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacb3b7-1fc6-4d3a-b98a-98619de367b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.271602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacb3b7-1fc6-4d3a-b98a-98619de367b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.271881 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacb3b7-1fc6-4d3a-b98a-98619de367b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.272513 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacb3b7-1fc6-4d3a-b98a-98619de367b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.273541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WARNING py.warnings [None req-7cacb3b7-1fc6-4d3a-b98a-98619de367b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:51:57.273541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: warnings.warn(msg) Aug 24 03:51:57.273541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: Aug 24 03:51:57.274822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacb3b7-1fc6-4d3a-b98a-98619de367b6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.304703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 775/1574] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:57.304703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422df5d-93cd-450d-9229-874f1484b344 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.363957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422df5d-93cd-450d-9229-874f1484b344 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.370259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495091dc-c483-4b94-94a0-626134e3cff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.370910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495091dc-c483-4b94-94a0-626134e3cff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.371192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495091dc-c483-4b94-94a0-626134e3cff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.371477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495091dc-c483-4b94-94a0-626134e3cff0 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.372106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495091dc-c483-4b94-94a0-626134e3cff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.374190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495091dc-c483-4b94-94a0-626134e3cff0 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.453756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 800/1575] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:57.454616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacb3b7-1fc6-4d3a-b98a-98619de367b6 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.465672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacb3b7-1fc6-4d3a-b98a-98619de367b6 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.471135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06554fe9-e616-4363-81de-deb6e4d9e213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.471775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06554fe9-e616-4363-81de-deb6e4d9e213 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.472055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06554fe9-e616-4363-81de-deb6e4d9e213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.472334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06554fe9-e616-4363-81de-deb6e4d9e213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.472954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06554fe9-e616-4363-81de-deb6e4d9e213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.474906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06554fe9-e616-4363-81de-deb6e4d9e213 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.585907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 776/1576] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/users => generated 285 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:57.587009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495091dc-c483-4b94-94a0-626134e3cff0 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.597430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495091dc-c483-4b94-94a0-626134e3cff0 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.602293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7869b6-4f15-4383-8eff-99a7385bc9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.602945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7869b6-4f15-4383-8eff-99a7385bc9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.603226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7869b6-4f15-4383-8eff-99a7385bc9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.603499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7869b6-4f15-4383-8eff-99a7385bc9d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.604126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7869b6-4f15-4383-8eff-99a7385bc9d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.606104 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7869b6-4f15-4383-8eff-99a7385bc9d1 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.616619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 801/1577] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:57.626420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06554fe9-e616-4363-81de-deb6e4d9e213 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.682170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06554fe9-e616-4363-81de-deb6e4d9e213 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.687069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5fddd-df26-4762-ad74-9f842a171014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.688598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5fddd-df26-4762-ad74-9f842a171014 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.688873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5fddd-df26-4762-ad74-9f842a171014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.689151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5fddd-df26-4762-ad74-9f842a171014 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.689787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5fddd-df26-4762-ad74-9f842a171014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.691941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5fddd-df26-4762-ad74-9f842a171014 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.704469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 777/1578] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:57.705437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7869b6-4f15-4383-8eff-99a7385bc9d1 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.715039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7869b6-4f15-4383-8eff-99a7385bc9d1 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.720635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7d79c-c379-4c61-9959-a0351600bcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.721222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7d79c-c379-4c61-9959-a0351600bcf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.721500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7d79c-c379-4c61-9959-a0351600bcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.721777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7d79c-c379-4c61-9959-a0351600bcf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.722388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7d79c-c379-4c61-9959-a0351600bcf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.724343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7d79c-c379-4c61-9959-a0351600bcf3 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.802184 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 802/1579] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:57.803561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2885e17-5827-4f7f-a954-0bb9da1d3d1d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.803854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2885e17-5827-4f7f-a954-0bb9da1d3d1d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.804129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2885e17-5827-4f7f-a954-0bb9da1d3d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.829175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e2885e17-5827-4f7f-a954-0bb9da1d3d1d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:57.830405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e2885e17-5827-4f7f-a954-0bb9da1d3d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:57.834852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 778/1580] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:57.834852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7d79c-c379-4c61-9959-a0351600bcf3 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.840751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 803/1581] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:57.840751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2885e17-5827-4f7f-a954-0bb9da1d3d1d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.847884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7d79c-c379-4c61-9959-a0351600bcf3 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.851444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2885e17-5827-4f7f-a954-0bb9da1d3d1d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.853276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822274b-f212-439f-8b82-fa3f3bf0586b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.853900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822274b-f212-439f-8b82-fa3f3bf0586b admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.854183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822274b-f212-439f-8b82-fa3f3bf0586b admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.854458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822274b-f212-439f-8b82-fa3f3bf0586b admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.855130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822274b-f212-439f-8b82-fa3f3bf0586b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.856143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428876bd-6c0a-4ef6-a0da-6115d7a6c8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.856750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428876bd-6c0a-4ef6-a0da-6115d7a6c8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.857037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428876bd-6c0a-4ef6-a0da-6115d7a6c8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.857185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822274b-f212-439f-8b82-fa3f3bf0586b admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.857353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428876bd-6c0a-4ef6-a0da-6115d7a6c8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.858015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428876bd-6c0a-4ef6-a0da-6115d7a6c8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.859961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428876bd-6c0a-4ef6-a0da-6115d7a6c8b8 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.870852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 804/1582] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:57.871666 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428876bd-6c0a-4ef6-a0da-6115d7a6c8b8 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.881513 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428876bd-6c0a-4ef6-a0da-6115d7a6c8b8 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.886096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353ee93-6204-4123-980d-545908b1d330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.886724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353ee93-6204-4123-980d-545908b1d330 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.887007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353ee93-6204-4123-980d-545908b1d330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.887285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353ee93-6204-4123-980d-545908b1d330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.887916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353ee93-6204-4123-980d-545908b1d330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.889532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353ee93-6204-4123-980d-545908b1d330 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.900634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 805/1583] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:57.901426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353ee93-6204-4123-980d-545908b1d330 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.911928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353ee93-6204-4123-980d-545908b1d330 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.916958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2125f-10dc-4d58-b6bb-dcb0d7ee5df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.917576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2125f-10dc-4d58-b6bb-dcb0d7ee5df9 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.927299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2125f-10dc-4d58-b6bb-dcb0d7ee5df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.927299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2125f-10dc-4d58-b6bb-dcb0d7ee5df9 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.927299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2125f-10dc-4d58-b6bb-dcb0d7ee5df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.927299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2125f-10dc-4d58-b6bb-dcb0d7ee5df9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.957119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 779/1584] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:57.962441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822274b-f212-439f-8b82-fa3f3bf0586b admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.967674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822274b-f212-439f-8b82-fa3f3bf0586b admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:57.973145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a77b7-292f-46f2-ae88-788813376aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:57.974354 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a77b7-292f-46f2-ae88-788813376aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:57.974354 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a77b7-292f-46f2-ae88-788813376aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:57.974354 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a77b7-292f-46f2-ae88-788813376aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:57.976620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a77b7-292f-46f2-ae88-788813376aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:57.976620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a77b7-292f-46f2-ae88-788813376aa2 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:57.987951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 780/1585] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:57.991371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a77b7-292f-46f2-ae88-788813376aa2 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:57.999281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a77b7-292f-46f2-ae88-788813376aa2 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.004736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c13019-926f-48dd-8311-205cf664783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.008485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c13019-926f-48dd-8311-205cf664783f admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.008485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c13019-926f-48dd-8311-205cf664783f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.008485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c13019-926f-48dd-8311-205cf664783f admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.008485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c13019-926f-48dd-8311-205cf664783f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.008485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c13019-926f-48dd-8311-205cf664783f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.022897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 781/1586] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.022897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c13019-926f-48dd-8311-205cf664783f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.030840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c13019-926f-48dd-8311-205cf664783f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.035609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4972f0-9bd6-41c9-8775-7c453d99c03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.036382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4972f0-9bd6-41c9-8775-7c453d99c03a admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.037000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4972f0-9bd6-41c9-8775-7c453d99c03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.037000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4972f0-9bd6-41c9-8775-7c453d99c03a admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.037596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4972f0-9bd6-41c9-8775-7c453d99c03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.039309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4972f0-9bd6-41c9-8775-7c453d99c03a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.049055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 806/1587] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:57 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:58.049901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2125f-10dc-4d58-b6bb-dcb0d7ee5df9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.050723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 782/1588] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.051689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4972f0-9bd6-41c9-8775-7c453d99c03a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.054988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-23d2125f-10dc-4d58-b6bb-dcb0d7ee5df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:58.066138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4972f0-9bd6-41c9-8775-7c453d99c03a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.068723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:58.071319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275801e3-e89f-4043-9b4c-65fc5a7a4670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.071941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275801e3-e89f-4043-9b4c-65fc5a7a4670 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.072226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275801e3-e89f-4043-9b4c-65fc5a7a4670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.072652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275801e3-e89f-4043-9b4c-65fc5a7a4670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.073315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275801e3-e89f-4043-9b4c-65fc5a7a4670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.075168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275801e3-e89f-4043-9b4c-65fc5a7a4670 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.087708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 783/1589] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.088965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275801e3-e89f-4043-9b4c-65fc5a7a4670 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.100322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275801e3-e89f-4043-9b4c-65fc5a7a4670 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.105136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec94530-51ec-4ffe-9128-c57e9a5f96ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.105758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec94530-51ec-4ffe-9128-c57e9a5f96ea admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.106421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec94530-51ec-4ffe-9128-c57e9a5f96ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.106421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec94530-51ec-4ffe-9128-c57e9a5f96ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.107803 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec94530-51ec-4ffe-9128-c57e9a5f96ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.109618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec94530-51ec-4ffe-9128-c57e9a5f96ea admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.124238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 784/1590] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.124238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec94530-51ec-4ffe-9128-c57e9a5f96ea admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.125043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2125f-10dc-4d58-b6bb-dcb0d7ee5df9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.127736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba3519-735a-41e2-85c4-676a6d62a5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.128377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba3519-735a-41e2-85c4-676a6d62a5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.134627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba3519-735a-41e2-85c4-676a6d62a5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.134627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba3519-735a-41e2-85c4-676a6d62a5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.134816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec94530-51ec-4ffe-9128-c57e9a5f96ea admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.137953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518cdf0-ec70-4eca-a340-9c4b8002fe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.138498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba3519-735a-41e2-85c4-676a6d62a5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.138624 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518cdf0-ec70-4eca-a340-9c4b8002fe9f admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.138858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518cdf0-ec70-4eca-a340-9c4b8002fe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.139141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518cdf0-ec70-4eca-a340-9c4b8002fe9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.139782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518cdf0-ec70-4eca-a340-9c4b8002fe9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.140053 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba3519-735a-41e2-85c4-676a6d62a5a7 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.142067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518cdf0-ec70-4eca-a340-9c4b8002fe9f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.147526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:58.150603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 807/1591] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:58.151280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba3519-735a-41e2-85c4-676a6d62a5a7 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.152979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 785/1592] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.153842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518cdf0-ec70-4eca-a340-9c4b8002fe9f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.160898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba3519-735a-41e2-85c4-676a6d62a5a7 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.167991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518cdf0-ec70-4eca-a340-9c4b8002fe9f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.169123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696c110-ab94-441b-a2dc-8a38cd1a068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.169915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696c110-ab94-441b-a2dc-8a38cd1a068a admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.170209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696c110-ab94-441b-a2dc-8a38cd1a068a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.170612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696c110-ab94-441b-a2dc-8a38cd1a068a admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.172244 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696c110-ab94-441b-a2dc-8a38cd1a068a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.173708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd066c3-1358-445e-93f9-a214c29d546d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.173967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696c110-ab94-441b-a2dc-8a38cd1a068a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.174326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd066c3-1358-445e-93f9-a214c29d546d admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.174690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd066c3-1358-445e-93f9-a214c29d546d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.175114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd066c3-1358-445e-93f9-a214c29d546d admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.175924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd066c3-1358-445e-93f9-a214c29d546d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.177533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd066c3-1358-445e-93f9-a214c29d546d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.186688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 808/1593] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.192144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696c110-ab94-441b-a2dc-8a38cd1a068a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.192384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 786/1594] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.192384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd066c3-1358-445e-93f9-a214c29d546d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.198705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696c110-ab94-441b-a2dc-8a38cd1a068a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.201813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd066c3-1358-445e-93f9-a214c29d546d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.204116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b20cdd-0c04-41e0-a83e-822167da4605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.204739 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20cdd-0c04-41e0-a83e-822167da4605 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.205024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20cdd-0c04-41e0-a83e-822167da4605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.205300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20cdd-0c04-41e0-a83e-822167da4605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.205946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b20cdd-0c04-41e0-a83e-822167da4605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.207490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518455f-1052-4f03-beda-c6482988693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.207630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b20cdd-0c04-41e0-a83e-822167da4605 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.208092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518455f-1052-4f03-beda-c6482988693f admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.208369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518455f-1052-4f03-beda-c6482988693f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.208651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518455f-1052-4f03-beda-c6482988693f admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.209365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518455f-1052-4f03-beda-c6482988693f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.211020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518455f-1052-4f03-beda-c6482988693f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.217830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 809/1595] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.218856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b20cdd-0c04-41e0-a83e-822167da4605 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.225367 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 787/1596] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.232267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518455f-1052-4f03-beda-c6482988693f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.232405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b20cdd-0c04-41e0-a83e-822167da4605 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.236779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084b789-885e-4f7b-a8fe-6e503cd8f135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.237234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518455f-1052-4f03-beda-c6482988693f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.237462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084b789-885e-4f7b-a8fe-6e503cd8f135 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.237748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084b789-885e-4f7b-a8fe-6e503cd8f135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.238025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084b789-885e-4f7b-a8fe-6e503cd8f135 admin admin] PATH_INFO: `/v3/projects/46ef5b02d8da4d2eac26b6ccfb35e1a4/users/e5582d10ebef459084fe287c18568f98/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.241987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc84376-315f-4003-9bf3-19fdc861f5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.242700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc84376-315f-4003-9bf3-19fdc861f5c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.242986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc84376-315f-4003-9bf3-19fdc861f5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.243263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc84376-315f-4003-9bf3-19fdc861f5c3 admin admin] PATH_INFO: `/v3/projects/c371c030d6354e94bf5de4a8864b7268/users/e4e92ef0339743a0ab3553b37d708a74/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.254225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084b789-885e-4f7b-a8fe-6e503cd8f135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ef5b02d8da4d2eac26b6ccfb35e1a4, user_id=e5582d10ebef459084fe287c18568f98, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.256838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084b789-885e-4f7b-a8fe-6e503cd8f135 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.259892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc84376-315f-4003-9bf3-19fdc861f5c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c371c030d6354e94bf5de4a8864b7268, user_id=e4e92ef0339743a0ab3553b37d708a74, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.260951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WARNING py.warnings [None req-cbc84376-315f-4003-9bf3-19fdc861f5c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:51:58.260951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: warnings.warn(msg) Aug 24 03:51:58.260951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: Aug 24 03:51:58.263280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc84376-315f-4003-9bf3-19fdc861f5c3 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.364023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 810/1597] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:58 2022] PUT /identity/v3/projects/46ef5b02d8da4d2eac26b6ccfb35e1a4/users/e5582d10ebef459084fe287c18568f98/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:58.364023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084b789-885e-4f7b-a8fe-6e503cd8f135 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.423303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084b789-885e-4f7b-a8fe-6e503cd8f135 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.427897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 788/1598] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:58 2022] PUT /identity/v3/projects/c371c030d6354e94bf5de4a8864b7268/users/e4e92ef0339743a0ab3553b37d708a74/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:58.432750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc84376-315f-4003-9bf3-19fdc861f5c3 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.496257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc84376-315f-4003-9bf3-19fdc861f5c3 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.499148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222b1b4-ab3d-4dbd-a184-10f474e43cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.499892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222b1b4-ab3d-4dbd-a184-10f474e43cfd admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.500177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222b1b4-ab3d-4dbd-a184-10f474e43cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.501516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222b1b4-ab3d-4dbd-a184-10f474e43cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.501852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222b1b4-ab3d-4dbd-a184-10f474e43cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.502982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989b9a2-d616-4dbc-891c-ab2d2db54d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.504083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222b1b4-ab3d-4dbd-a184-10f474e43cfd admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.504213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989b9a2-d616-4dbc-891c-ab2d2db54d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.504737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989b9a2-d616-4dbc-891c-ab2d2db54d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.505035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989b9a2-d616-4dbc-891c-ab2d2db54d9d admin admin] PATH_INFO: `/v3/projects/feac5069d37149628113087d9d93912b/users/87a0f4d8b8c341ed85618e70adb2def4/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.524126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989b9a2-d616-4dbc-891c-ab2d2db54d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=feac5069d37149628113087d9d93912b, user_id=87a0f4d8b8c341ed85618e70adb2def4, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.526677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 811/1599] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.529628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989b9a2-d616-4dbc-891c-ab2d2db54d9d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.529729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222b1b4-ab3d-4dbd-a184-10f474e43cfd admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.544006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222b1b4-ab3d-4dbd-a184-10f474e43cfd admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.550389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626faa2c-6da0-4a9e-b107-ac1265836ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.550820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626faa2c-6da0-4a9e-b107-ac1265836ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.551115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626faa2c-6da0-4a9e-b107-ac1265836ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.551390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626faa2c-6da0-4a9e-b107-ac1265836ba7 admin admin] PATH_INFO: `/v3/projects/63408551e1b24a86b8f0059f1a948896/users/547bce0d8133462c95bcb6f272776a27/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.554813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:58.568984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626faa2c-6da0-4a9e-b107-ac1265836ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63408551e1b24a86b8f0059f1a948896, user_id=547bce0d8133462c95bcb6f272776a27, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.571719 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626faa2c-6da0-4a9e-b107-ac1265836ba7 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.654093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 789/1600] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:58 2022] PUT /identity/v3/projects/feac5069d37149628113087d9d93912b/users/87a0f4d8b8c341ed85618e70adb2def4/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:58.654987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989b9a2-d616-4dbc-891c-ab2d2db54d9d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.678342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 812/1601] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:58 2022] PUT /identity/v3/projects/63408551e1b24a86b8f0059f1a948896/users/547bce0d8133462c95bcb6f272776a27/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:58.679203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626faa2c-6da0-4a9e-b107-ac1265836ba7 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.720879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989b9a2-d616-4dbc-891c-ab2d2db54d9d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.745829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626faa2c-6da0-4a9e-b107-ac1265836ba7 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.754166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0740a-04d1-45c3-9eae-b386e7b5c25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.755957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0740a-04d1-45c3-9eae-b386e7b5c25f admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.757873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0740a-04d1-45c3-9eae-b386e7b5c25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.758023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0740a-04d1-45c3-9eae-b386e7b5c25f admin admin] PATH_INFO: `/v3/projects/1015969e512341fc99783bc6dcd472f3/users/c10d31651ceb45e2940a53b10f034a0a/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.776404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0740a-04d1-45c3-9eae-b386e7b5c25f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1015969e512341fc99783bc6dcd472f3, user_id=c10d31651ceb45e2940a53b10f034a0a, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.779377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0740a-04d1-45c3-9eae-b386e7b5c25f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.779702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:58.799758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec7343-f8ec-485a-8c7f-aa0af7f07b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.800163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec7343-f8ec-485a-8c7f-aa0af7f07b72 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.801383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec7343-f8ec-485a-8c7f-aa0af7f07b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.801668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec7343-f8ec-485a-8c7f-aa0af7f07b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.802699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec7343-f8ec-485a-8c7f-aa0af7f07b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.804328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec7343-f8ec-485a-8c7f-aa0af7f07b72 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.816629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 790/1602] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:58.817786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3e1a4-2652-4e89-a38a-c763ec94bc9d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.818080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3e1a4-2652-4e89-a38a-c763ec94bc9d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.818362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3e1a4-2652-4e89-a38a-c763ec94bc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.840167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-1ac3e1a4-2652-4e89-a38a-c763ec94bc9d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:58.842404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1ac3e1a4-2652-4e89-a38a-c763ec94bc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:58.850746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 791/1603] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:58.852171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22894a-4005-40f9-8723-7779e731aec1 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.852466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22894a-4005-40f9-8723-7779e731aec1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.852744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22894a-4005-40f9-8723-7779e731aec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.872555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 813/1604] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:58 2022] PUT /identity/v3/projects/1015969e512341fc99783bc6dcd472f3/users/c10d31651ceb45e2940a53b10f034a0a/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:58.879871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0740a-04d1-45c3-9eae-b386e7b5c25f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:58.895815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7b22894a-4005-40f9-8723-7779e731aec1 None None] MFA Rules not processed for user `e4e92ef0339743a0ab3553b37d708a74`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:58.897046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7b22894a-4005-40f9-8723-7779e731aec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:58.946976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0740a-04d1-45c3-9eae-b386e7b5c25f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:58.951541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959a865-da15-4c3a-b9b8-5206de244a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:58.954732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959a865-da15-4c3a-b9b8-5206de244a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.954732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959a865-da15-4c3a-b9b8-5206de244a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.954732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959a865-da15-4c3a-b9b8-5206de244a40 admin admin] PATH_INFO: `/v3/projects/798e09b25ee042b49d1df467b70d4fc4/users/0158925721c1446483b19f9957425732/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.969539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 792/1605] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:58.969539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12aafb-4c94-4027-b576-f515bc7de0a0 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:58.969539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12aafb-4c94-4027-b576-f515bc7de0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:58.969539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12aafb-4c94-4027-b576-f515bc7de0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:58.972436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959a865-da15-4c3a-b9b8-5206de244a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798e09b25ee042b49d1df467b70d4fc4, user_id=0158925721c1446483b19f9957425732, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:58.975363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959a865-da15-4c3a-b9b8-5206de244a40 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:58.990887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-5a12aafb-4c94-4027-b576-f515bc7de0a0 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:58.992343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5a12aafb-4c94-4027-b576-f515bc7de0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:59.000947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 793/1606] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:59.002190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32821c59-a493-472c-b78c-abe69560e360 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.002471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32821c59-a493-472c-b78c-abe69560e360 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.002777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32821c59-a493-472c-b78c-abe69560e360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.029213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-32821c59-a493-472c-b78c-abe69560e360 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:59.030691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-32821c59-a493-472c-b78c-abe69560e360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:59.039194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 794/1607] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:59.040518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32821c59-a493-472c-b78c-abe69560e360 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.053423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32821c59-a493-472c-b78c-abe69560e360 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.059180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d8da7-eafb-4302-9e4e-0eb6d36e05a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:59.060007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d8da7-eafb-4302-9e4e-0eb6d36e05a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.060409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d8da7-eafb-4302-9e4e-0eb6d36e05a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.060796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d8da7-eafb-4302-9e4e-0eb6d36e05a9 admin admin] PATH_INFO: `/v3/projects/bdb802f52e1140d488ff0c6d739cd0d1/users/c7e29563153a4ecbbe7447bed0324df7/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.078262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d8da7-eafb-4302-9e4e-0eb6d36e05a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb802f52e1140d488ff0c6d739cd0d1, user_id=c7e29563153a4ecbbe7447bed0324df7, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:59.081025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d8da7-eafb-4302-9e4e-0eb6d36e05a9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:59.098995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 814/1608] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:58 2022] PUT /identity/v3/projects/798e09b25ee042b49d1df467b70d4fc4/users/0158925721c1446483b19f9957425732/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:59.101313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959a865-da15-4c3a-b9b8-5206de244a40 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.158406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959a865-da15-4c3a-b9b8-5206de244a40 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.162293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31444071-c21a-4350-b872-3771fe32327e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:59.162942 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31444071-c21a-4350-b872-3771fe32327e service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.163254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31444071-c21a-4350-b872-3771fe32327e service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.163533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31444071-c21a-4350-b872-3771fe32327e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.168051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:59.184702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 795/1609] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:59 2022] PUT /identity/v3/projects/bdb802f52e1140d488ff0c6d739cd0d1/users/c7e29563153a4ecbbe7447bed0324df7/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:59.186272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86342d-63fd-42b6-ad6a-c499db7f40f7 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.186609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86342d-63fd-42b6-ad6a-c499db7f40f7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.187429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86342d-63fd-42b6-ad6a-c499db7f40f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.217375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-fc86342d-63fd-42b6-ad6a-c499db7f40f7 None None] MFA Rules not processed for user `c10d31651ceb45e2940a53b10f034a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:59.218587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fc86342d-63fd-42b6-ad6a-c499db7f40f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:59.225378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31444071-c21a-4350-b872-3771fe32327e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:59.226990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31444071-c21a-4350-b872-3771fe32327e service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:59.237533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 815/1610] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:59.238428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31444071-c21a-4350-b872-3771fe32327e service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.248051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31444071-c21a-4350-b872-3771fe32327e service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.292271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 796/1611] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:59.293626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86342d-63fd-42b6-ad6a-c499db7f40f7 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.320691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6dfd4-9937-4594-8998-066d683f7fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:59.320807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:59.320900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6dfd4-9937-4594-8998-066d683f7fad admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.320900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6dfd4-9937-4594-8998-066d683f7fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.320900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6dfd4-9937-4594-8998-066d683f7fad admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.320900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6dfd4-9937-4594-8998-066d683f7fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:59.322113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6dfd4-9937-4594-8998-066d683f7fad admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:59.367345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86342d-63fd-42b6-ad6a-c499db7f40f7 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.371235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95f05f-3363-4d4c-b16a-22acf20e3903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:59.374196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95f05f-3363-4d4c-b16a-22acf20e3903 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.374196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95f05f-3363-4d4c-b16a-22acf20e3903 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.374196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95f05f-3363-4d4c-b16a-22acf20e3903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.383647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95f05f-3363-4d4c-b16a-22acf20e3903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:59.385111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95f05f-3363-4d4c-b16a-22acf20e3903 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:59.396001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 797/1612] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:59.396900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95f05f-3363-4d4c-b16a-22acf20e3903 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.404810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95f05f-3363-4d4c-b16a-22acf20e3903 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.411066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be4080-2196-4ce9-8c17-dbeb4eeadf89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:59.411692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be4080-2196-4ce9-8c17-dbeb4eeadf89 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.412003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be4080-2196-4ce9-8c17-dbeb4eeadf89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.412295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be4080-2196-4ce9-8c17-dbeb4eeadf89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.423725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be4080-2196-4ce9-8c17-dbeb4eeadf89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:59.425235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be4080-2196-4ce9-8c17-dbeb4eeadf89 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:59.430609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 816/1613] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:59 2022] POST /identity/v3/projects => generated 369 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:59.430609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f437-a7eb-4d09-9aeb-b9bc1983e5a1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.430609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f437-a7eb-4d09-9aeb-b9bc1983e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.430609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f437-a7eb-4d09-9aeb-b9bc1983e5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.438771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 798/1614] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:59.440430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c218b0-310b-4b90-a13b-81b761361cab None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.440793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c218b0-310b-4b90-a13b-81b761361cab None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.441393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c218b0-310b-4b90-a13b-81b761361cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.462134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e3c218b0-310b-4b90-a13b-81b761361cab None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:59.466600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e3c218b0-310b-4b90-a13b-81b761361cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:59.466744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-55c7f437-a7eb-4d09-9aeb-b9bc1983e5a1 None None] MFA Rules not processed for user `0158925721c1446483b19f9957425732`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:59.466744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-55c7f437-a7eb-4d09-9aeb-b9bc1983e5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:59.532491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 799/1615] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:59.535952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d982d0-041b-4bf1-9427-3edfbe8f525e None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.536515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d982d0-041b-4bf1-9427-3edfbe8f525e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.536804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d982d0-041b-4bf1-9427-3edfbe8f525e None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.538470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 800/1616] 10.222.0.82 () {54 vars in 962 bytes} [Wed Aug 24 03:51:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:51:59.539293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d982d0-041b-4bf1-9427-3edfbe8f525e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.558825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 817/1617] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:51:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:59.558825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f437-a7eb-4d09-9aeb-b9bc1983e5a1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.570486 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d982d0-041b-4bf1-9427-3edfbe8f525e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.582801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057a5ad-a577-4f8f-a94b-ce1ad59bbab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:59.583969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a5ad-a577-4f8f-a94b-ce1ad59bbab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.584256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a5ad-a577-4f8f-a94b-ce1ad59bbab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.584539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a5ad-a577-4f8f-a94b-ce1ad59bbab0 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.585188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057a5ad-a577-4f8f-a94b-ce1ad59bbab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:59.588168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057a5ad-a577-4f8f-a94b-ce1ad59bbab0 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:59.612284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:59.685654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f437-a7eb-4d09-9aeb-b9bc1983e5a1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.690149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d98eb-964f-4f10-aabc-6e68ec354896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:59.691277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d98eb-964f-4f10-aabc-6e68ec354896 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.691569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d98eb-964f-4f10-aabc-6e68ec354896 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.691846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d98eb-964f-4f10-aabc-6e68ec354896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.706453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d98eb-964f-4f10-aabc-6e68ec354896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:59.708511 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d98eb-964f-4f10-aabc-6e68ec354896 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:59.711252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 801/1618] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:59 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:59.712986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bfaf7-897f-4d14-be8a-2f9692bc19ec None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.713586 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bfaf7-897f-4d14-be8a-2f9692bc19ec None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.714122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bfaf7-897f-4d14-be8a-2f9692bc19ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.724726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 818/1619] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:59.726121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bc766-0475-4c3a-8549-0d6b3c6774ee None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.726428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bc766-0475-4c3a-8549-0d6b3c6774ee None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.726889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bc766-0475-4c3a-8549-0d6b3c6774ee None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.736268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 819/1620] 10.222.0.82 () {54 vars in 962 bytes} [Wed Aug 24 03:51:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:51:59.737393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bc766-0475-4c3a-8549-0d6b3c6774ee None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.756122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bc766-0475-4c3a-8549-0d6b3c6774ee None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.768121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6942c-31f7-426a-a74c-6abc9bffc93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:59.769063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6942c-31f7-426a-a74c-6abc9bffc93d admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.769362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6942c-31f7-426a-a74c-6abc9bffc93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.769642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6942c-31f7-426a-a74c-6abc9bffc93d admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.770306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6942c-31f7-426a-a74c-6abc9bffc93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:59.772355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6942c-31f7-426a-a74c-6abc9bffc93d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:59.786044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 820/1621] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:59.789134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fe75b-de3d-407f-b574-a3f35ef30fc3 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.789443 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fe75b-de3d-407f-b574-a3f35ef30fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.789726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fe75b-de3d-407f-b574-a3f35ef30fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.803570 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-a94bfaf7-897f-4d14-be8a-2f9692bc19ec None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:59.810391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a94bfaf7-897f-4d14-be8a-2f9692bc19ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:59.832062 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e60fe75b-de3d-407f-b574-a3f35ef30fc3 None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:59.834247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e60fe75b-de3d-407f-b574-a3f35ef30fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:59.914630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 802/1622] 10.222.0.82 () {58 vars in 1059 bytes} [Wed Aug 24 03:51:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:59.916249 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bfaf7-897f-4d14-be8a-2f9692bc19ec None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.931151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bfaf7-897f-4d14-be8a-2f9692bc19ec None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.938085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60363e-8b78-47f1-aa23-338db71d3764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:59.939257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60363e-8b78-47f1-aa23-338db71d3764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:59.939723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60363e-8b78-47f1-aa23-338db71d3764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:59.940168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60363e-8b78-47f1-aa23-338db71d3764 admin admin] PATH_INFO: `/v3/projects/ad82a396ac274baf8f4d76b64e9693c4/users/c5f1127e04024518a76432819624aecb/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:59.964881 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 821/1623] 10.222.0.82 () {58 vars in 1059 bytes} [Wed Aug 24 03:51:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:59.966185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fe75b-de3d-407f-b574-a3f35ef30fc3 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:59.976087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:59.981850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60363e-8b78-47f1-aa23-338db71d3764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad82a396ac274baf8f4d76b64e9693c4, user_id=c5f1127e04024518a76432819624aecb, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:59.985813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fe75b-de3d-407f-b574-a3f35ef30fc3 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:59.986275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60363e-8b78-47f1-aa23-338db71d3764 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:00.011747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47362621-5e7a-4e0f-9175-1703af14f033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:00.012448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47362621-5e7a-4e0f-9175-1703af14f033 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.012738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47362621-5e7a-4e0f-9175-1703af14f033 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.013014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47362621-5e7a-4e0f-9175-1703af14f033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.058809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 803/1624] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:59 2022] PUT /identity/v3/projects/ad82a396ac274baf8f4d76b64e9693c4/users/c5f1127e04024518a76432819624aecb/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:00.061116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60363e-8b78-47f1-aa23-338db71d3764 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:00.143696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47362621-5e7a-4e0f-9175-1703af14f033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:00.146625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47362621-5e7a-4e0f-9175-1703af14f033 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:00.173332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 822/1625] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:00.176159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47362621-5e7a-4e0f-9175-1703af14f033 service nova] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:00.199349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47362621-5e7a-4e0f-9175-1703af14f033 service nova] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:00.242941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60363e-8b78-47f1-aa23-338db71d3764 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:00.250741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5592d-8032-4d3a-b9d4-a9b53f4db6d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:00.251473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5592d-8032-4d3a-b9d4-a9b53f4db6d4 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.251758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5592d-8032-4d3a-b9d4-a9b53f4db6d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.252036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5592d-8032-4d3a-b9d4-a9b53f4db6d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.307876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa6ce8-2da6-4192-9f71-2c4164d89af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:00.310247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa6ce8-2da6-4192-9f71-2c4164d89af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.311693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa6ce8-2da6-4192-9f71-2c4164d89af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.312626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa6ce8-2da6-4192-9f71-2c4164d89af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.314675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa6ce8-2da6-4192-9f71-2c4164d89af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:00.319793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa6ce8-2da6-4192-9f71-2c4164d89af6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:00.326443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5592d-8032-4d3a-b9d4-a9b53f4db6d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:00.329216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5592d-8032-4d3a-b9d4-a9b53f4db6d4 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:00.347628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 823/1626] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:00.351830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa6ce8-2da6-4192-9f71-2c4164d89af6 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:00.362319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 804/1627] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:00.382717 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa6ce8-2da6-4192-9f71-2c4164d89af6 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:00.394034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa947c95-755b-45f6-8353-1cba14c7fbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:00.394672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa947c95-755b-45f6-8353-1cba14c7fbcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.395054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa947c95-755b-45f6-8353-1cba14c7fbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.395456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa947c95-755b-45f6-8353-1cba14c7fbcc admin admin] PATH_INFO: `/v3/projects/ad82a396ac274baf8f4d76b64e9693c4/users/c5f1127e04024518a76432819624aecb/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.402297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa947c95-755b-45f6-8353-1cba14c7fbcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad82a396ac274baf8f4d76b64e9693c4, user_id=c5f1127e04024518a76432819624aecb, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:00.406754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa947c95-755b-45f6-8353-1cba14c7fbcc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:00.433367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 824/1628] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:00 2022] PUT /identity/v3/projects/ad82a396ac274baf8f4d76b64e9693c4/users/c5f1127e04024518a76432819624aecb/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:00.445264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd405f03-a8f7-4a7c-9774-735e63872b82 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.445693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd405f03-a8f7-4a7c-9774-735e63872b82 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.445977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd405f03-a8f7-4a7c-9774-735e63872b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.511198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-bd405f03-a8f7-4a7c-9774-735e63872b82 None None] MFA Rules not processed for user `c5f1127e04024518a76432819624aecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:00.516211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bd405f03-a8f7-4a7c-9774-735e63872b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:00.572926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:00.587220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4ab1f-8323-493e-8124-f2b5025d13e1 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.587945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4ab1f-8323-493e-8124-f2b5025d13e1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.589136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4ab1f-8323-493e-8124-f2b5025d13e1 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.593794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 825/1629] 10.222.0.82 () {54 vars in 912 bytes} [Wed Aug 24 03:52:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:52:00.603246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ecec1-906f-4c17-95e8-afe0b2a72837 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.609307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ecec1-906f-4c17-95e8-afe0b2a72837 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.609307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ecec1-906f-4c17-95e8-afe0b2a72837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.643261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 805/1630] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:00.655025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd405f03-a8f7-4a7c-9774-735e63872b82 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:00.675238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5e8ecec1-906f-4c17-95e8-afe0b2a72837 None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:00.676596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5e8ecec1-906f-4c17-95e8-afe0b2a72837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:00.760392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd405f03-a8f7-4a7c-9774-735e63872b82 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:00.767442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e889c2d-0c67-4636-8a9b-b78be186a5f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:00.768728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889c2d-0c67-4636-8a9b-b78be186a5f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.769147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889c2d-0c67-4636-8a9b-b78be186a5f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.769491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889c2d-0c67-4636-8a9b-b78be186a5f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.794665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e889c2d-0c67-4636-8a9b-b78be186a5f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:00.794665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e889c2d-0c67-4636-8a9b-b78be186a5f1 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:00.801983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 806/1631] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:52:00.833195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 826/1632] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:52:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:00.848767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e889c2d-0c67-4636-8a9b-b78be186a5f1 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:00.855811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b41b66-67c4-483f-9c5b-29e88fd37aa0 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.856324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b41b66-67c4-483f-9c5b-29e88fd37aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.857251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b41b66-67c4-483f-9c5b-29e88fd37aa0 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.859017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 827/1633] 10.222.0.82 () {54 vars in 912 bytes} [Wed Aug 24 03:52:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:52:00.861255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e889c2d-0c67-4636-8a9b-b78be186a5f1 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:00.864893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0748d-8241-4e9a-b365-c23fc4524bd8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.865458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0748d-8241-4e9a-b365-c23fc4524bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.865950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0748d-8241-4e9a-b365-c23fc4524bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.866414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5dc1-59ab-40b8-a183-e42e7e8bf231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:00.867209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5dc1-59ab-40b8-a183-e42e7e8bf231 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:00.867499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5dc1-59ab-40b8-a183-e42e7e8bf231 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:00.867781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5dc1-59ab-40b8-a183-e42e7e8bf231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:00.886423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5dc1-59ab-40b8-a183-e42e7e8bf231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:01.271130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5dc1-59ab-40b8-a183-e42e7e8bf231 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:01.274020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a9f0748d-8241-4e9a-b365-c23fc4524bd8 None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:01.274020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a9f0748d-8241-4e9a-b365-c23fc4524bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:01.274020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 828/1634] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:52:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:01.277380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 807/1635] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:01.283305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0748d-8241-4e9a-b365-c23fc4524bd8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:01.283305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0748d-8241-4e9a-b365-c23fc4524bd8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:01.283305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5d7d7-7a7b-410c-9093-0b97ea242f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:01.283305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5d7d7-7a7b-410c-9093-0b97ea242f09 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:01.289022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5d7d7-7a7b-410c-9093-0b97ea242f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:01.289022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5d7d7-7a7b-410c-9093-0b97ea242f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:01.289022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5d7d7-7a7b-410c-9093-0b97ea242f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:01.289022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5d7d7-7a7b-410c-9093-0b97ea242f09 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:01.292395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:01.293978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 829/1636] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:01.294410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5dc1-59ab-40b8-a183-e42e7e8bf231 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:01.294410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5dc1-59ab-40b8-a183-e42e7e8bf231 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:01.294410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6815c23-22ad-4254-bf5e-83697869c84b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:01.294410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6815c23-22ad-4254-bf5e-83697869c84b service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:01.295257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6815c23-22ad-4254-bf5e-83697869c84b service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:01.295257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6815c23-22ad-4254-bf5e-83697869c84b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:01.363228 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6815c23-22ad-4254-bf5e-83697869c84b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:01.367459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6815c23-22ad-4254-bf5e-83697869c84b service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:01.382252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 808/1637] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:52:02.631637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5d7d7-7a7b-410c-9093-0b97ea242f09 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:02.847345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5d7d7-7a7b-410c-9093-0b97ea242f09 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:02.863989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:02.866249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:02.866466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:02.867866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] PATH_INFO: `/v3/users/0158925721c1446483b19f9957425732` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:02.870922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0158925721c1446483b19f9957425732)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:02.877122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:03.144374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:03.156534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0158925721c1446483b19f9957425732'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:03.157231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0158925721c1446483b19f9957425732'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:03.241018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0158925721c1446483b19f9957425732'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:03.254005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0158925721c1446483b19f9957425732'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:03.271469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0158925721c1446483b19f9957425732'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:03.282759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 830/1638] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:02 2022] DELETE /identity/v3/users/0158925721c1446483b19f9957425732 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:03.288112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d1a0-4e3d-430a-b2bd-26989c6b7afc None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:03.289436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d1a0-4e3d-430a-b2bd-26989c6b7afc None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:03.290046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d1a0-4e3d-430a-b2bd-26989c6b7afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:03.325466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-cd36d1a0-4e3d-430a-b2bd-26989c6b7afc None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:03.327787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cd36d1a0-4e3d-430a-b2bd-26989c6b7afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:03.415613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 809/1639] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:03.454137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:03.455594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:03.561839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79c847-32c0-431b-ae2a-83499ae855d4 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:03.568822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97685a69-3006-45d5-8dea-d2dd440c597d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:03.569760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97685a69-3006-45d5-8dea-d2dd440c597d service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:03.570233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97685a69-3006-45d5-8dea-d2dd440c597d service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:03.570745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97685a69-3006-45d5-8dea-d2dd440c597d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:03.595889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97685a69-3006-45d5-8dea-d2dd440c597d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:03.601041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97685a69-3006-45d5-8dea-d2dd440c597d service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:03.643794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 831/1640] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:04.869048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36d1a0-4e3d-430a-b2bd-26989c6b7afc None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:04.870278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cd36d1a0-4e3d-430a-b2bd-26989c6b7afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:04.928022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36d1a0-4e3d-430a-b2bd-26989c6b7afc None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:04.947890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:04.948558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:04.948844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:04.949120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] PATH_INFO: `/v3/projects/798e09b25ee042b49d1df467b70d4fc4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:04.950325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=798e09b25ee042b49d1df467b70d4fc4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:04.953401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:05.108619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:05.161678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798e09b25ee042b49d1df467b70d4fc4', 'request_id': 'req-c2c20428-114a-4eba-ac54-556cb3882241'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:05.293724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798e09b25ee042b49d1df467b70d4fc4', 'request_id': 'req-c2c20428-114a-4eba-ac54-556cb3882241'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:05.333368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798e09b25ee042b49d1df467b70d4fc4', 'request_id': 'req-c2c20428-114a-4eba-ac54-556cb3882241'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:05.385138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] The token cache is being invalidate because project 798e09b25ee042b49d1df467b70d4fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:05.385874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:05.386279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2c20428-114a-4eba-ac54-556cb3882241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:05.388565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 810/1641] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:04 2022] DELETE /identity/v3/projects/798e09b25ee042b49d1df467b70d4fc4 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:05.393824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6ddf-2187-4291-9d7b-910d73f876e3 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:05.394328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6ddf-2187-4291-9d7b-910d73f876e3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:05.394412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6ddf-2187-4291-9d7b-910d73f876e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:05.431613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-677b6ddf-2187-4291-9d7b-910d73f876e3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:05.440182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-677b6ddf-2187-4291-9d7b-910d73f876e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:05.632943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 832/1642] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:05.643376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a069bc4-3a60-4bfa-ac06-595f6d600cc0 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:05.643751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a069bc4-3a60-4bfa-ac06-595f6d600cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:05.644035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a069bc4-3a60-4bfa-ac06-595f6d600cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:05.681489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-5a069bc4-3a60-4bfa-ac06-595f6d600cc0 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:05.685959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5a069bc4-3a60-4bfa-ac06-595f6d600cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:05.718674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6ddf-2187-4291-9d7b-910d73f876e3 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:05.723924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 811/1643] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:05.738851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6ddf-2187-4291-9d7b-910d73f876e3 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:05.758388 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f637b-61b2-4990-89e5-34090170f671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:05.759199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f637b-61b2-4990-89e5-34090170f671 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:05.759554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f637b-61b2-4990-89e5-34090170f671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:05.759896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f637b-61b2-4990-89e5-34090170f671 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:05.760675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f637b-61b2-4990-89e5-34090170f671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:05.761249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f637b-61b2-4990-89e5-34090170f671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:05.763131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f637b-61b2-4990-89e5-34090170f671 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:05.793469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 833/1644] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:52:05.800080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a069bc4-3a60-4bfa-ac06-595f6d600cc0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:05.823664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a069bc4-3a60-4bfa-ac06-595f6d600cc0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:05.844241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171b014-e74d-4206-894e-1670d5b284ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:05.844912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171b014-e74d-4206-894e-1670d5b284ac admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:05.845199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171b014-e74d-4206-894e-1670d5b284ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:05.849263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171b014-e74d-4206-894e-1670d5b284ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:05.852841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171b014-e74d-4206-894e-1670d5b284ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:05.855058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171b014-e74d-4206-894e-1670d5b284ac admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:06.012038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 812/1645] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:05 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:06.016571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f637b-61b2-4990-89e5-34090170f671 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:06.039196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f637b-61b2-4990-89e5-34090170f671 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:06.145320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:06.217203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847edc20-bcf0-4427-b06d-48f05e2408ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:06.217203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847edc20-bcf0-4427-b06d-48f05e2408ea admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:06.217203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847edc20-bcf0-4427-b06d-48f05e2408ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:06.217203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847edc20-bcf0-4427-b06d-48f05e2408ea admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:06.221604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847edc20-bcf0-4427-b06d-48f05e2408ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:06.225683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847edc20-bcf0-4427-b06d-48f05e2408ea admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:06.334497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 834/1646] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:06 2022] POST /identity/v3/users => generated 290 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:52:06.338765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171b014-e74d-4206-894e-1670d5b284ac admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:06.356026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171b014-e74d-4206-894e-1670d5b284ac admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:06.371374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644fd2c-5b8c-40d7-865d-9eea3e204a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:06.372553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644fd2c-5b8c-40d7-865d-9eea3e204a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:06.375461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644fd2c-5b8c-40d7-865d-9eea3e204a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:06.380190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644fd2c-5b8c-40d7-865d-9eea3e204a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:06.380190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644fd2c-5b8c-40d7-865d-9eea3e204a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:06.380190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644fd2c-5b8c-40d7-865d-9eea3e204a3a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:06.405645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 813/1647] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:52:06.408812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847edc20-bcf0-4427-b06d-48f05e2408ea admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:06.426117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847edc20-bcf0-4427-b06d-48f05e2408ea admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:06.437356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7580fd-dc16-4539-b05c-cebe27f51390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:06.438476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7580fd-dc16-4539-b05c-cebe27f51390 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:06.439015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7580fd-dc16-4539-b05c-cebe27f51390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:06.440545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7580fd-dc16-4539-b05c-cebe27f51390 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:06.441449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7580fd-dc16-4539-b05c-cebe27f51390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:06.443240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7580fd-dc16-4539-b05c-cebe27f51390 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:06.481110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 835/1648] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:06.487165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644fd2c-5b8c-40d7-865d-9eea3e204a3a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:06.500903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644fd2c-5b8c-40d7-865d-9eea3e204a3a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:06.525252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c954c3e-1e60-4778-816c-82d9e754267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:06.525252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c954c3e-1e60-4778-816c-82d9e754267c admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:06.526704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c954c3e-1e60-4778-816c-82d9e754267c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:06.526704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c954c3e-1e60-4778-816c-82d9e754267c admin admin] PATH_INFO: `/v3/projects/9300ca7749774c998eeb2e3cfa01bcd1/users/1a644976db314ec38ca785c2b7112fb1/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:06.575917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c954c3e-1e60-4778-816c-82d9e754267c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9300ca7749774c998eeb2e3cfa01bcd1, user_id=1a644976db314ec38ca785c2b7112fb1, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:06.575917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c954c3e-1e60-4778-816c-82d9e754267c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:06.710851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 814/1649] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:06 2022] PUT /identity/v3/projects/9300ca7749774c998eeb2e3cfa01bcd1/users/1a644976db314ec38ca785c2b7112fb1/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:06.721249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e525e8-031b-48f7-b00e-f0ed69bcefea None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:06.721670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e525e8-031b-48f7-b00e-f0ed69bcefea None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:06.722052 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e525e8-031b-48f7-b00e-f0ed69bcefea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:06.784962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-96e525e8-031b-48f7-b00e-f0ed69bcefea None None] MFA Rules not processed for user `1a644976db314ec38ca785c2b7112fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:06.786527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-96e525e8-031b-48f7-b00e-f0ed69bcefea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:06.878208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c954c3e-1e60-4778-816c-82d9e754267c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:06.884032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2c954c3e-1e60-4778-816c-82d9e754267c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:06.972465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 836/1650] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:06.974334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e525e8-031b-48f7-b00e-f0ed69bcefea None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:07.080933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c954c3e-1e60-4778-816c-82d9e754267c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:07.098958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:07.100032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:07.100570 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:07.100665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] PATH_INFO: `/v3/users/c10d31651ceb45e2940a53b10f034a0a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:07.102262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10d31651ceb45e2940a53b10f034a0a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:07.103361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: WARNING py.warnings [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:52:07.103361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: warnings.warn(msg) Aug 24 03:52:07.103361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: Aug 24 03:52:07.104610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:07.276109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e525e8-031b-48f7-b00e-f0ed69bcefea None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:07.299891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73178b37-2353-45ee-a0fd-156608dcec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:07.300766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73178b37-2353-45ee-a0fd-156608dcec89 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:07.301064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73178b37-2353-45ee-a0fd-156608dcec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:07.301526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73178b37-2353-45ee-a0fd-156608dcec89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:07.302179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73178b37-2353-45ee-a0fd-156608dcec89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:07.304302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73178b37-2353-45ee-a0fd-156608dcec89 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:07.449961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 837/1651] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:06 2022] POST /identity/v3/projects => generated 379 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:07.451986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73178b37-2353-45ee-a0fd-156608dcec89 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:07.464724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10d31651ceb45e2940a53b10f034a0a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:07.543019 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10d31651ceb45e2940a53b10f034a0a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:07.546580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:07.595181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73178b37-2353-45ee-a0fd-156608dcec89 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:07.597779 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10d31651ceb45e2940a53b10f034a0a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:07.617829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70f760-820c-4237-8a1d-9f52d17ba840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:07.618577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70f760-820c-4237-8a1d-9f52d17ba840 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:07.618935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70f760-820c-4237-8a1d-9f52d17ba840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:07.619281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70f760-820c-4237-8a1d-9f52d17ba840 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:07.620387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70f760-820c-4237-8a1d-9f52d17ba840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:07.625256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70f760-820c-4237-8a1d-9f52d17ba840 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:07.636307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10d31651ceb45e2940a53b10f034a0a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:07.637873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c2d4e6db-e165-4e70-a599-000e62c719a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10d31651ceb45e2940a53b10f034a0a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:07.658156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 815/1652] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:06 2022] DELETE /identity/v3/users/c10d31651ceb45e2940a53b10f034a0a => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:07.658533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80ad8-24ab-479e-b3f9-38d95dd5a0db None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:07.658992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80ad8-24ab-479e-b3f9-38d95dd5a0db None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:07.660295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80ad8-24ab-479e-b3f9-38d95dd5a0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:07.699972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-dfc80ad8-24ab-479e-b3f9-38d95dd5a0db None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:07.701543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dfc80ad8-24ab-479e-b3f9-38d95dd5a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:07.749446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 838/1653] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:07 2022] POST /identity/v3/users => generated 296 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:52:07.753197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70f760-820c-4237-8a1d-9f52d17ba840 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:07.763156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3e70f760-820c-4237-8a1d-9f52d17ba840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:07.783225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70f760-820c-4237-8a1d-9f52d17ba840 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:07.832333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 816/1654] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:07.844938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc80ad8-24ab-479e-b3f9-38d95dd5a0db None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:07.846684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dfc80ad8-24ab-479e-b3f9-38d95dd5a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:07.917741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75970008-7511-4910-89ed-c37f2ccd0a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:07.918495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75970008-7511-4910-89ed-c37f2ccd0a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:07.919685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75970008-7511-4910-89ed-c37f2ccd0a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:07.919905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75970008-7511-4910-89ed-c37f2ccd0a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:07.920734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75970008-7511-4910-89ed-c37f2ccd0a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:07.927114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75970008-7511-4910-89ed-c37f2ccd0a6c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:07.946873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 839/1655] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:52:07.952266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75970008-7511-4910-89ed-c37f2ccd0a6c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:07.975979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75970008-7511-4910-89ed-c37f2ccd0a6c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:07.989748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c3db2-fb44-4adf-ad5a-2da8d855dfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:07.990699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c3db2-fb44-4adf-ad5a-2da8d855dfd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:07.990805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c3db2-fb44-4adf-ad5a-2da8d855dfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:07.991158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c3db2-fb44-4adf-ad5a-2da8d855dfd6 admin admin] PATH_INFO: `/v3/projects/d78f36597b2d4497a94b7e85a38c6a40/users/08173118774341f2abce9abaad176220/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:08.004255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc80ad8-24ab-479e-b3f9-38d95dd5a0db None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:08.025046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c3db2-fb44-4adf-ad5a-2da8d855dfd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78f36597b2d4497a94b7e85a38c6a40, user_id=08173118774341f2abce9abaad176220, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:08.029860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c3db2-fb44-4adf-ad5a-2da8d855dfd6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:08.031432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d54a2-785b-4bf9-9ef6-86c9a70df61f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:08.034278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d54a2-785b-4bf9-9ef6-86c9a70df61f service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:08.034791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d54a2-785b-4bf9-9ef6-86c9a70df61f service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:08.035294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d54a2-785b-4bf9-9ef6-86c9a70df61f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:08.071746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d54a2-785b-4bf9-9ef6-86c9a70df61f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:08.082748 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d54a2-785b-4bf9-9ef6-86c9a70df61f service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:08.127301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 817/1656] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:08.201914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 840/1657] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:07 2022] PUT /identity/v3/projects/d78f36597b2d4497a94b7e85a38c6a40/users/08173118774341f2abce9abaad176220/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:08.216113 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d54a2-785b-4bf9-9ef6-86c9a70df61f service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:08.239517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d54a2-785b-4bf9-9ef6-86c9a70df61f service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:08.391103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fee090-5499-494c-ac0d-801619158131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:08.391324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fee090-5499-494c-ac0d-801619158131 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:08.392928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fee090-5499-494c-ac0d-801619158131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:08.393244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fee090-5499-494c-ac0d-801619158131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:08.393896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fee090-5499-494c-ac0d-801619158131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:08.395594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fee090-5499-494c-ac0d-801619158131 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:08.412582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 818/1658] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:08.416909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c3db2-fb44-4adf-ad5a-2da8d855dfd6 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:08.429206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c3db2-fb44-4adf-ad5a-2da8d855dfd6 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:08.436063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585c8bb-8e9f-47ba-9858-a972a420bd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:08.436900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585c8bb-8e9f-47ba-9858-a972a420bd06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:08.437311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585c8bb-8e9f-47ba-9858-a972a420bd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:08.437604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585c8bb-8e9f-47ba-9858-a972a420bd06 admin admin] PATH_INFO: `/v3/projects/d78f36597b2d4497a94b7e85a38c6a40/users/08173118774341f2abce9abaad176220/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:08.440287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585c8bb-8e9f-47ba-9858-a972a420bd06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78f36597b2d4497a94b7e85a38c6a40, user_id=08173118774341f2abce9abaad176220, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:08.443385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585c8bb-8e9f-47ba-9858-a972a420bd06 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:08.478879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 841/1659] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:08 2022] PUT /identity/v3/projects/d78f36597b2d4497a94b7e85a38c6a40/users/08173118774341f2abce9abaad176220/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:08.484445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156267e-5dc3-4ed7-9e37-318ac92337b2 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:08.484790 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156267e-5dc3-4ed7-9e37-318ac92337b2 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:08.485144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156267e-5dc3-4ed7-9e37-318ac92337b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:08.586806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2156267e-5dc3-4ed7-9e37-318ac92337b2 None None] MFA Rules not processed for user `08173118774341f2abce9abaad176220`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:08.588714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2156267e-5dc3-4ed7-9e37-318ac92337b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:08.640505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:08.721054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 819/1660] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:08.752342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585c8bb-8e9f-47ba-9858-a972a420bd06 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:08.754724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d585c8bb-8e9f-47ba-9858-a972a420bd06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:08.976059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585c8bb-8e9f-47ba-9858-a972a420bd06 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:08.988964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418bf2c-f1eb-41dd-a47b-8ad3ecfb5f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:08.989636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418bf2c-f1eb-41dd-a47b-8ad3ecfb5f1f service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:08.990050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418bf2c-f1eb-41dd-a47b-8ad3ecfb5f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:08.990334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418bf2c-f1eb-41dd-a47b-8ad3ecfb5f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:09.025535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418bf2c-f1eb-41dd-a47b-8ad3ecfb5f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:09.029738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418bf2c-f1eb-41dd-a47b-8ad3ecfb5f1f service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:09.054557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 842/1661] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:09.260954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156267e-5dc3-4ed7-9e37-318ac92337b2 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:09.263427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2156267e-5dc3-4ed7-9e37-318ac92337b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:09.426003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156267e-5dc3-4ed7-9e37-318ac92337b2 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:09.431015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:09.432085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:09.432507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:09.434026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] PATH_INFO: `/v3/projects/1015969e512341fc99783bc6dcd472f3` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:09.436363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1015969e512341fc99783bc6dcd472f3)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:09.439685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:09.673842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418bf2c-f1eb-41dd-a47b-8ad3ecfb5f1f service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:09.675307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6418bf2c-f1eb-41dd-a47b-8ad3ecfb5f1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:09.702750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418bf2c-f1eb-41dd-a47b-8ad3ecfb5f1f service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:09.733198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:09.733198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:09.733198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:09.733198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] PATH_INFO: `/v3/users/e4e92ef0339743a0ab3553b37d708a74` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:09.734103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e92ef0339743a0ab3553b37d708a74)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:09.734103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:09.841864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1015969e512341fc99783bc6dcd472f3', 'request_id': 'req-45e6abbb-5da8-41d5-82c2-90727275f016'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.074097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1015969e512341fc99783bc6dcd472f3', 'request_id': 'req-45e6abbb-5da8-41d5-82c2-90727275f016'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.085294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1015969e512341fc99783bc6dcd472f3', 'request_id': 'req-45e6abbb-5da8-41d5-82c2-90727275f016'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.103282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] The token cache is being invalidate because project 1015969e512341fc99783bc6dcd472f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:10.103579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.104603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-45e6abbb-5da8-41d5-82c2-90727275f016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.109058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 820/1662] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:09 2022] DELETE /identity/v3/projects/1015969e512341fc99783bc6dcd472f3 => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:52:10.122370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef07b7e-9163-446c-9e6d-8ebf5cb3ea14 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:10.122693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef07b7e-9163-446c-9e6d-8ebf5cb3ea14 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:10.122973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef07b7e-9163-446c-9e6d-8ebf5cb3ea14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:10.176844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7ef07b7e-9163-446c-9e6d-8ebf5cb3ea14 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:10.178335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7ef07b7e-9163-446c-9e6d-8ebf5cb3ea14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:10.329669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 821/1663] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:10.331222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aa1d4-419e-4865-ba16-93f9d635fd5f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:10.331513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aa1d4-419e-4865-ba16-93f9d635fd5f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:10.331794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aa1d4-419e-4865-ba16-93f9d635fd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:10.340504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e92ef0339743a0ab3553b37d708a74'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.345915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e92ef0339743a0ab3553b37d708a74'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.384278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-8c2aa1d4-419e-4865-ba16-93f9d635fd5f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:10.385534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8c2aa1d4-419e-4865-ba16-93f9d635fd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:10.402398 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 822/1664] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:10.407541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aa1d4-419e-4865-ba16-93f9d635fd5f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:10.428293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aa1d4-419e-4865-ba16-93f9d635fd5f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:10.433872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacbdd5-0a08-4be2-bf9d-aa84c24a5240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:10.434497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacbdd5-0a08-4be2-bf9d-aa84c24a5240 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:10.434946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacbdd5-0a08-4be2-bf9d-aa84c24a5240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:10.442800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacbdd5-0a08-4be2-bf9d-aa84c24a5240 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:10.442800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aacbdd5-0a08-4be2-bf9d-aa84c24a5240 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:10.442800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aacbdd5-0a08-4be2-bf9d-aa84c24a5240 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:10.442800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aacbdd5-0a08-4be2-bf9d-aa84c24a5240 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:10.456508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 823/1665] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:52:10.461417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacbdd5-0a08-4be2-bf9d-aa84c24a5240 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:10.474397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacbdd5-0a08-4be2-bf9d-aa84c24a5240 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:10.483454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603338f8-7b1d-4868-b9a0-d7e84ae2a520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:10.484448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603338f8-7b1d-4868-b9a0-d7e84ae2a520 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:10.485050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603338f8-7b1d-4868-b9a0-d7e84ae2a520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:10.485520 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603338f8-7b1d-4868-b9a0-d7e84ae2a520 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:10.499945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603338f8-7b1d-4868-b9a0-d7e84ae2a520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:10.499945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603338f8-7b1d-4868-b9a0-d7e84ae2a520 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:10.583708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e92ef0339743a0ab3553b37d708a74'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.583708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e92ef0339743a0ab3553b37d708a74'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.592456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ee0e6d95-bd52-4ad1-a33e-34effba08ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e92ef0339743a0ab3553b37d708a74'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:10.603404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 843/1666] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:09 2022] DELETE /identity/v3/users/e4e92ef0339743a0ab3553b37d708a74 => generated 0 bytes in 930 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:10.607622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6634eaa-6a0d-4c3a-8730-d434f8172a3f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:10.608104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6634eaa-6a0d-4c3a-8730-d434f8172a3f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:10.608524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6634eaa-6a0d-4c3a-8730-d434f8172a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:10.632250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e6634eaa-6a0d-4c3a-8730-d434f8172a3f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:10.633778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e6634eaa-6a0d-4c3a-8730-d434f8172a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:10.775149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 844/1667] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:10.784258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 824/1668] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:10 2022] POST /identity/v3/projects => generated 405 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:10.787595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603338f8-7b1d-4868-b9a0-d7e84ae2a520 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:10.788309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6634eaa-6a0d-4c3a-8730-d434f8172a3f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:10.793574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-603338f8-7b1d-4868-b9a0-d7e84ae2a520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:10.806299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6634eaa-6a0d-4c3a-8730-d434f8172a3f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:10.949028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ea4b8-5489-4185-a7ee-23cad7e0d94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:10.960295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea4b8-5489-4185-a7ee-23cad7e0d94b admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:10.960295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea4b8-5489-4185-a7ee-23cad7e0d94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:10.960295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea4b8-5489-4185-a7ee-23cad7e0d94b admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:10.960295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ea4b8-5489-4185-a7ee-23cad7e0d94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:10.960791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603338f8-7b1d-4868-b9a0-d7e84ae2a520 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:10.961418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ea4b8-5489-4185-a7ee-23cad7e0d94b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:10.967022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91131b-f5fd-42a8-b349-c422f10cb56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:10.967709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91131b-f5fd-42a8-b349-c422f10cb56e service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:10.967994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91131b-f5fd-42a8-b349-c422f10cb56e service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:10.968274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91131b-f5fd-42a8-b349-c422f10cb56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:10.998258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91131b-f5fd-42a8-b349-c422f10cb56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:11.006832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91131b-f5fd-42a8-b349-c422f10cb56e service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:11.046402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 825/1669] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:11.273036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91131b-f5fd-42a8-b349-c422f10cb56e service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:11.283439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 845/1670] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:10 2022] POST /identity/v3/users => generated 303 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:11.285979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91131b-f5fd-42a8-b349-c422f10cb56e service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:11.298432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a0485-f2e8-4415-85be-e5c7736c0434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:11.299207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a0485-f2e8-4415-85be-e5c7736c0434 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:11.300014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a0485-f2e8-4415-85be-e5c7736c0434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:11.300554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a0485-f2e8-4415-85be-e5c7736c0434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:11.301363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a0485-f2e8-4415-85be-e5c7736c0434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:11.303806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a0485-f2e8-4415-85be-e5c7736c0434 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:11.319268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 826/1671] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:11.323104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ea4b8-5489-4185-a7ee-23cad7e0d94b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:11.346018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ea4b8-5489-4185-a7ee-23cad7e0d94b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:11.375548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc1f7e-1f23-4c05-b662-da86c463fc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:11.377259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc1f7e-1f23-4c05-b662-da86c463fc6d admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:11.378930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc1f7e-1f23-4c05-b662-da86c463fc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:11.379204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc1f7e-1f23-4c05-b662-da86c463fc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:11.379911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc1f7e-1f23-4c05-b662-da86c463fc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:11.381910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc1f7e-1f23-4c05-b662-da86c463fc6d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:11.422978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 846/1672] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:11.426934 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a0485-f2e8-4415-85be-e5c7736c0434 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:11.444472 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a0485-f2e8-4415-85be-e5c7736c0434 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:11.466443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550169e0-450a-4351-803b-c54b0cc1917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:11.467185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550169e0-450a-4351-803b-c54b0cc1917c admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:11.478391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550169e0-450a-4351-803b-c54b0cc1917c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:11.478391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550169e0-450a-4351-803b-c54b0cc1917c admin admin] PATH_INFO: `/v3/projects/e04af3a5c44f4efd833ee8866d3e1fb4/users/70ae06180a5949cf8114791c4d9ebf47/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:11.541215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550169e0-450a-4351-803b-c54b0cc1917c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04af3a5c44f4efd833ee8866d3e1fb4, user_id=70ae06180a5949cf8114791c4d9ebf47, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:11.549314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550169e0-450a-4351-803b-c54b0cc1917c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:11.677410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc1f7e-1f23-4c05-b662-da86c463fc6d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:11.680098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-edfc1f7e-1f23-4c05-b662-da86c463fc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:11.691578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 827/1673] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:11 2022] PUT /identity/v3/projects/e04af3a5c44f4efd833ee8866d3e1fb4/users/70ae06180a5949cf8114791c4d9ebf47/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:11.705039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f9e02-2ca8-4cf7-b67d-2c634d785532 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:11.705513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f9e02-2ca8-4cf7-b67d-2c634d785532 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:11.705927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f9e02-2ca8-4cf7-b67d-2c634d785532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:11.764299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-fd7f9e02-2ca8-4cf7-b67d-2c634d785532 None None] MFA Rules not processed for user `70ae06180a5949cf8114791c4d9ebf47`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:11.765773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fd7f9e02-2ca8-4cf7-b67d-2c634d785532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:11.858975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:11.861923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc1f7e-1f23-4c05-b662-da86c463fc6d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:11.966667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 828/1674] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:11 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:11.967813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f9e02-2ca8-4cf7-b67d-2c634d785532 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:12.044684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f459440-97f5-4b81-ab7e-f5741658c77c tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08173118774341f2abce9abaad176220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd78f36597b2d4497a94b7e85a38c6a40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:12.045346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f459440-97f5-4b81-ab7e-f5741658c77c tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:12.045637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f459440-97f5-4b81-ab7e-f5741658c77c tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:12.045925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f459440-97f5-4b81-ab7e-f5741658c77c tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:12.046635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f459440-97f5-4b81-ab7e-f5741658c77c tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:12.047127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f459440-97f5-4b81-ab7e-f5741658c77c tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:12.048788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f459440-97f5-4b81-ab7e-f5741658c77c tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:12.088632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 847/1675] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:52:12.091091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f459440-97f5-4b81-ab7e-f5741658c77c tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:12.121865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f459440-97f5-4b81-ab7e-f5741658c77c tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:12.126761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f9e02-2ca8-4cf7-b67d-2c634d785532 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:12.130272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41931e85-eb59-45c1-bf80-17afcf54e155 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08173118774341f2abce9abaad176220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd78f36597b2d4497a94b7e85a38c6a40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:12.136733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41931e85-eb59-45c1-bf80-17afcf54e155 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:12.136872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54b8f2-7840-4737-aeed-3d052ee86a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:12.136980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41931e85-eb59-45c1-bf80-17afcf54e155 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:12.136980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41931e85-eb59-45c1-bf80-17afcf54e155 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:12.137125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54b8f2-7840-4737-aeed-3d052ee86a89 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:12.137252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41931e85-eb59-45c1-bf80-17afcf54e155 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:12.137373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54b8f2-7840-4737-aeed-3d052ee86a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:12.137373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54b8f2-7840-4737-aeed-3d052ee86a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:12.137373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54b8f2-7840-4737-aeed-3d052ee86a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:12.143819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41931e85-eb59-45c1-bf80-17afcf54e155 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:12.145620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54b8f2-7840-4737-aeed-3d052ee86a89 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:12.319569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:12.484411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 848/1676] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:12 2022] POST /identity/v3/projects => generated 355 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:12.488627 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 829/1677] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:11 2022] POST /identity/v3/projects => generated 403 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:12.489655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41931e85-eb59-45c1-bf80-17afcf54e155 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:12.490694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54b8f2-7840-4737-aeed-3d052ee86a89 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:12.643224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41931e85-eb59-45c1-bf80-17afcf54e155 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:12.677334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54b8f2-7840-4737-aeed-3d052ee86a89 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:12.695043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aef67-1dcd-42be-bb2d-3fd42ae9a93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:12.697309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aef67-1dcd-42be-bb2d-3fd42ae9a93c admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:12.697997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aef67-1dcd-42be-bb2d-3fd42ae9a93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:12.698818 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5aef67-1dcd-42be-bb2d-3fd42ae9a93c admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:12.700652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aef67-1dcd-42be-bb2d-3fd42ae9a93c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:12.709953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5aef67-1dcd-42be-bb2d-3fd42ae9a93c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:12.760113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08173118774341f2abce9abaad176220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd78f36597b2d4497a94b7e85a38c6a40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:12.760945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:12.765792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:12.766469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:12.767404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:12.767762 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:12.770448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:12.788854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 849/1678] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:52:12.790063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:12.795654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:12.827427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef8723-3778-42c2-af11-2f49458110b1 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:12.835291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:12.835961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:12.836867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:12.836867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] PATH_INFO: `/v3/projects/c371c030d6354e94bf5de4a8864b7268` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:12.837711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c371c030d6354e94bf5de4a8864b7268)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:12.838804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: WARNING py.warnings [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:52:12.838804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: warnings.warn(msg) Aug 24 03:52:12.838804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: Aug 24 03:52:12.840987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:13.039568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 830/1679] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:12 2022] POST /identity/v3/users => generated 308 bytes in 550 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:13.043750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aef67-1dcd-42be-bb2d-3fd42ae9a93c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:13.081261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5aef67-1dcd-42be-bb2d-3fd42ae9a93c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:13.098315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08173118774341f2abce9abaad176220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd78f36597b2d4497a94b7e85a38c6a40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:13.102719 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:13.103001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:13.104833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] PATH_INFO: `/v3/projects/5290b3b832624e38a7a86510546281d2` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:13.112305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5290b3b832624e38a7a86510546281d2)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:13.118924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:13.187288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c371c030d6354e94bf5de4a8864b7268', 'request_id': 'req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.206821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c371c030d6354e94bf5de4a8864b7268', 'request_id': 'req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.352525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c371c030d6354e94bf5de4a8864b7268', 'request_id': 'req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.384245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5290b3b832624e38a7a86510546281d2', 'request_id': 'req-fa8776c2-75de-4056-a5bc-a491b239c935'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.397025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] The token cache is being invalidate because project c371c030d6354e94bf5de4a8864b7268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:13.397419 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.398082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.399747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 850/1680] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:12 2022] DELETE /identity/v3/projects/c371c030d6354e94bf5de4a8864b7268 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:13.401153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:13.402065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:13.498680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73535d5-cfd6-47ae-82fe-fc1bb462a0d6 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:13.591087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb529d5-022b-499e-ab25-1165c2740b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:13.591865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb529d5-022b-499e-ab25-1165c2740b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:13.593365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb529d5-022b-499e-ab25-1165c2740b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:13.593365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb529d5-022b-499e-ab25-1165c2740b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:13.594561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb529d5-022b-499e-ab25-1165c2740b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:13.597183 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb529d5-022b-499e-ab25-1165c2740b3d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:13.616878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 851/1681] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:13.618539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb529d5-022b-499e-ab25-1165c2740b3d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:13.621294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5290b3b832624e38a7a86510546281d2', 'request_id': 'req-fa8776c2-75de-4056-a5bc-a491b239c935'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.625039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5cb529d5-022b-499e-ab25-1165c2740b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:13.635248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5290b3b832624e38a7a86510546281d2', 'request_id': 'req-fa8776c2-75de-4056-a5bc-a491b239c935'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.641652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb529d5-022b-499e-ab25-1165c2740b3d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:13.649593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:13.650477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:13.650914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:13.651302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] PATH_INFO: `/v3/users/c5f1127e04024518a76432819624aecb` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:13.654262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f1127e04024518a76432819624aecb)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:13.654706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] The token cache is being invalidate because project 5290b3b832624e38a7a86510546281d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:13.656710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:13.663153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.667093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:13.671109 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 831/1682] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:13 2022] DELETE /identity/v3/projects/5290b3b832624e38a7a86510546281d2 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:13.672255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:13.684605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:13.718022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8776c2-75de-4056-a5bc-a491b239c935 tempest-VolumeQuotaClassesTest-797319020 tempest-VolumeQuotaClassesTest-797319020-project-admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:13.723528 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdde88b-9164-4e49-9de7-909da3fee0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:13.724546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdde88b-9164-4e49-9de7-909da3fee0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:13.725039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdde88b-9164-4e49-9de7-909da3fee0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:13.725516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdde88b-9164-4e49-9de7-909da3fee0b9 admin admin] PATH_INFO: `/v3/projects/c5246b48c42a47e981a37316348f0558/users/8abb1033dd1e4e0ba9d482a9d139c90d/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:13.757632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdde88b-9164-4e49-9de7-909da3fee0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5246b48c42a47e981a37316348f0558, user_id=8abb1033dd1e4e0ba9d482a9d139c90d, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:13.762251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdde88b-9164-4e49-9de7-909da3fee0b9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:13.946018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 832/1683] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:13 2022] PUT /identity/v3/projects/c5246b48c42a47e981a37316348f0558/users/8abb1033dd1e4e0ba9d482a9d139c90d/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:13.949462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdde88b-9164-4e49-9de7-909da3fee0b9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:14.052952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f1127e04024518a76432819624aecb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:14.052952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f1127e04024518a76432819624aecb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:14.072874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdde88b-9164-4e49-9de7-909da3fee0b9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:14.080326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfd031-7b8c-4c4a-b791-8c3fefe03ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:14.087433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd031-7b8c-4c4a-b791-8c3fefe03ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:14.087433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd031-7b8c-4c4a-b791-8c3fefe03ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:14.087433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfd031-7b8c-4c4a-b791-8c3fefe03ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:14.087433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfd031-7b8c-4c4a-b791-8c3fefe03ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:14.087433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfd031-7b8c-4c4a-b791-8c3fefe03ca2 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:14.108414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 833/1684] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:14.112489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfd031-7b8c-4c4a-b791-8c3fefe03ca2 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:14.117092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-48bfd031-7b8c-4c4a-b791-8c3fefe03ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:14.141003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfd031-7b8c-4c4a-b791-8c3fefe03ca2 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:14.146050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc00fc-a3fa-42e3-a6d8-5fbf44e5e112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:14.146783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc00fc-a3fa-42e3-a6d8-5fbf44e5e112 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:14.147072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc00fc-a3fa-42e3-a6d8-5fbf44e5e112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:14.147353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc00fc-a3fa-42e3-a6d8-5fbf44e5e112 admin admin] PATH_INFO: `/v3/projects/c5246b48c42a47e981a37316348f0558/users/8abb1033dd1e4e0ba9d482a9d139c90d/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:14.152966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc00fc-a3fa-42e3-a6d8-5fbf44e5e112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5246b48c42a47e981a37316348f0558, user_id=8abb1033dd1e4e0ba9d482a9d139c90d, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:14.157253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc00fc-a3fa-42e3-a6d8-5fbf44e5e112 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:14.175192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 834/1685] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:14 2022] PUT /identity/v3/projects/c5246b48c42a47e981a37316348f0558/users/8abb1033dd1e4e0ba9d482a9d139c90d/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:14.188499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e11f4-1d89-41bc-8476-462dc0bbb01c None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:14.189727 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e11f4-1d89-41bc-8476-462dc0bbb01c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:14.189727 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e11f4-1d89-41bc-8476-462dc0bbb01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:14.197324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f1127e04024518a76432819624aecb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:14.206390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f1127e04024518a76432819624aecb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:14.238486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f1127e04024518a76432819624aecb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:14.241466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e94e11f4-1d89-41bc-8476-462dc0bbb01c None None] MFA Rules not processed for user `8abb1033dd1e4e0ba9d482a9d139c90d`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:14.244174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e94e11f4-1d89-41bc-8476-462dc0bbb01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:14.253777 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 852/1686] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:13 2022] DELETE /identity/v3/users/c5f1127e04024518a76432819624aecb => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:14.446081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 835/1687] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:14.457916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:14.459408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:14.607943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13875d3f-92ea-4a24-8b21-aff970c48199 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:14.611859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc28968-1677-4d27-b785-cf138b742457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:14.612500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc28968-1677-4d27-b785-cf138b742457 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:14.612784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc28968-1677-4d27-b785-cf138b742457 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:14.613078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc28968-1677-4d27-b785-cf138b742457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:14.768001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc28968-1677-4d27-b785-cf138b742457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:14.769481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc28968-1677-4d27-b785-cf138b742457 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:14.800457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 853/1688] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:14 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:52:15.084835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:15.181401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e11f4-1d89-41bc-8476-462dc0bbb01c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:15.183831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e94e11f4-1d89-41bc-8476-462dc0bbb01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:15.416709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e11f4-1d89-41bc-8476-462dc0bbb01c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:15.566737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:15.567449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:15.569417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:15.569417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] PATH_INFO: `/v3/users/1a644976db314ec38ca785c2b7112fb1` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:15.576397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a644976db314ec38ca785c2b7112fb1)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:15.576692 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc28968-1677-4d27-b785-cf138b742457 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:15.583256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:15.591547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc28968-1677-4d27-b785-cf138b742457 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:15.615840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:15.616537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:15.616830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:15.617112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] PATH_INFO: `/v3/users/70ae06180a5949cf8114791c4d9ebf47` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:15.618530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70ae06180a5949cf8114791c4d9ebf47)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:15.622046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:15.921632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a644976db314ec38ca785c2b7112fb1'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.034132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a644976db314ec38ca785c2b7112fb1'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.038734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70ae06180a5949cf8114791c4d9ebf47'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.040304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70ae06180a5949cf8114791c4d9ebf47'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.065204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a644976db314ec38ca785c2b7112fb1'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.079395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a644976db314ec38ca785c2b7112fb1'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.080189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a9a9e04f-1c55-41f0-8d98-10821d02495e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a644976db314ec38ca785c2b7112fb1'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.089686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 836/1689] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:15 2022] DELETE /identity/v3/users/1a644976db314ec38ca785c2b7112fb1 => generated 0 bytes in 909 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:16.091289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bfd26-034d-4f1f-ba9e-bd4f156ea035 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:16.091586 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bfd26-034d-4f1f-ba9e-bd4f156ea035 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:16.091871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bfd26-034d-4f1f-ba9e-bd4f156ea035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:16.150064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:16.156477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-380bfd26-034d-4f1f-ba9e-bd4f156ea035 None None] MFA Rules not processed for user `a466f40eba2a4975bd13614474ff527c`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:16.156477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-380bfd26-034d-4f1f-ba9e-bd4f156ea035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:16.196958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70ae06180a5949cf8114791c4d9ebf47'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.210028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70ae06180a5949cf8114791c4d9ebf47'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.249417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-09940cf2-98ee-4812-b450-791af6c2ca79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70ae06180a5949cf8114791c4d9ebf47'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:16.277425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 854/1690] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:15 2022] DELETE /identity/v3/users/70ae06180a5949cf8114791c4d9ebf47 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:16.280561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 837/1691] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:16.281380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd0af1-6809-4b06-9610-52b21e4eb820 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:16.282046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd0af1-6809-4b06-9610-52b21e4eb820 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:16.282731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab25d52-711e-4729-89b2-ba01db674a6c None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:16.283093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab25d52-711e-4729-89b2-ba01db674a6c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:16.283297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd0af1-6809-4b06-9610-52b21e4eb820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:16.283762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab25d52-711e-4729-89b2-ba01db674a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:16.331226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-fab25d52-711e-4729-89b2-ba01db674a6c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:16.332771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fab25d52-711e-4729-89b2-ba01db674a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:16.356277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9bcd0af1-6809-4b06-9610-52b21e4eb820 None None] MFA Rules not processed for user `87a0f4d8b8c341ed85618e70adb2def4`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:16.359285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9bcd0af1-6809-4b06-9610-52b21e4eb820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:16.586419 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 855/1692] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:16.591239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d06200-9420-42f3-9495-bb6fd388c350 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:16.591780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d06200-9420-42f3-9495-bb6fd388c350 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:16.593079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d06200-9420-42f3-9495-bb6fd388c350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:16.619229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 838/1693] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:16.620939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7ffcf-ff3c-4bd1-8802-4c8f7f95b7c3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:16.622996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7ffcf-ff3c-4bd1-8802-4c8f7f95b7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:16.623830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7ffcf-ff3c-4bd1-8802-4c8f7f95b7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:16.640054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-47d06200-9420-42f3-9495-bb6fd388c350 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:16.653401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-47d06200-9420-42f3-9495-bb6fd388c350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:16.675981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 856/1694] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:16.677680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2370030-4237-4083-9a85-30fb120e656a None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:16.679909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2370030-4237-4083-9a85-30fb120e656a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:16.681834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2370030-4237-4083-9a85-30fb120e656a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:16.702776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-fbe7ffcf-ff3c-4bd1-8802-4c8f7f95b7c3 None None] MFA Rules not processed for user `a466f40eba2a4975bd13614474ff527c`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:16.702776 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fbe7ffcf-ff3c-4bd1-8802-4c8f7f95b7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:16.753158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b2370030-4237-4083-9a85-30fb120e656a None None] MFA Rules not processed for user `547bce0d8133462c95bcb6f272776a27`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:16.754791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b2370030-4237-4083-9a85-30fb120e656a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:16.903714 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 857/1695] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:16.904945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2370030-4237-4083-9a85-30fb120e656a None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:16.912139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b2370030-4237-4083-9a85-30fb120e656a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:16.919934 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 839/1696] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:16.921115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7ffcf-ff3c-4bd1-8802-4c8f7f95b7c3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:16.924183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fbe7ffcf-ff3c-4bd1-8802-4c8f7f95b7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:17.052910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7ffcf-ff3c-4bd1-8802-4c8f7f95b7c3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:17.063652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851d6b-ef1e-41cc-b52d-6b325b03c241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:17.064501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851d6b-ef1e-41cc-b52d-6b325b03c241 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.064849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851d6b-ef1e-41cc-b52d-6b325b03c241 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.067640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851d6b-ef1e-41cc-b52d-6b325b03c241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:17.101214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851d6b-ef1e-41cc-b52d-6b325b03c241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:17.101763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:17.111298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851d6b-ef1e-41cc-b52d-6b325b03c241 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:17.137982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 840/1697] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:17.147382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851d6b-ef1e-41cc-b52d-6b325b03c241 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:17.154376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2370030-4237-4083-9a85-30fb120e656a None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:17.168910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851d6b-ef1e-41cc-b52d-6b325b03c241 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:17.175185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c3612-6dfc-40bb-a9f5-b8d3dcd01da7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:17.175988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c3612-6dfc-40bb-a9f5-b8d3dcd01da7 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.176275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c3612-6dfc-40bb-a9f5-b8d3dcd01da7 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.176551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c3612-6dfc-40bb-a9f5-b8d3dcd01da7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:17.182270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c398d74-48f0-4d6e-942e-d9306722d05e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:17.187383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c398d74-48f0-4d6e-942e-d9306722d05e service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.189784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c398d74-48f0-4d6e-942e-d9306722d05e service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.190264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c398d74-48f0-4d6e-942e-d9306722d05e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:17.206900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c3612-6dfc-40bb-a9f5-b8d3dcd01da7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:17.208563 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c3612-6dfc-40bb-a9f5-b8d3dcd01da7 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:17.218243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c398d74-48f0-4d6e-942e-d9306722d05e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:17.222756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c398d74-48f0-4d6e-942e-d9306722d05e service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:17.232301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 858/1698] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:17.234934 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c3612-6dfc-40bb-a9f5-b8d3dcd01da7 service nova] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:17.237490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b25c3612-6dfc-40bb-a9f5-b8d3dcd01da7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:17.252027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 841/1699] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:17.256689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd46c3-a652-4ede-b277-23e55b11a152 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.257603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd46c3-a652-4ede-b277-23e55b11a152 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.257974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd46c3-a652-4ede-b277-23e55b11a152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:17.273277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c3612-6dfc-40bb-a9f5-b8d3dcd01da7 service nova] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:17.292656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:17.295725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.297805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.298191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] PATH_INFO: `/v3/projects/ad82a396ac274baf8f4d76b64e9693c4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:17.300137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad82a396ac274baf8f4d76b64e9693c4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:17.311330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:17.316387 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-4fcd46c3-a652-4ede-b277-23e55b11a152 None None] MFA Rules not processed for user `547bce0d8133462c95bcb6f272776a27`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:17.320065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4fcd46c3-a652-4ede-b277-23e55b11a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:17.341373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 842/1700] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:17.342190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd46c3-a652-4ede-b277-23e55b11a152 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:17.347744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4fcd46c3-a652-4ede-b277-23e55b11a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:17.406117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd46c3-a652-4ede-b277-23e55b11a152 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:17.441187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c0e66-652b-48a0-982c-ed7a4faad834 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:17.443072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c0e66-652b-48a0-982c-ed7a4faad834 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.443368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c0e66-652b-48a0-982c-ed7a4faad834 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.443646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c0e66-652b-48a0-982c-ed7a4faad834 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:17.443956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad82a396ac274baf8f4d76b64e9693c4', 'request_id': 'req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:17.458343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad82a396ac274baf8f4d76b64e9693c4', 'request_id': 'req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:17.474746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c0e66-652b-48a0-982c-ed7a4faad834 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:17.478120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c0e66-652b-48a0-982c-ed7a4faad834 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:17.512310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 843/1701] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:17.545152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad82a396ac274baf8f4d76b64e9693c4', 'request_id': 'req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:17.574365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] The token cache is being invalidate because project ad82a396ac274baf8f4d76b64e9693c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:17.574700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:17.580423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:17.584426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 859/1702] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:17 2022] DELETE /identity/v3/projects/ad82a396ac274baf8f4d76b64e9693c4 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:17.591232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45647774-25fa-4c8c-aef9-b4422d1f9d66 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.591522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45647774-25fa-4c8c-aef9-b4422d1f9d66 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.591800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45647774-25fa-4c8c-aef9-b4422d1f9d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:17.650006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-45647774-25fa-4c8c-aef9-b4422d1f9d66 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:17.656206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:17.658024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:17.658124 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-45647774-25fa-4c8c-aef9-b4422d1f9d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:17.850826 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 844/1703] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:17.860091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c57d25-1769-472a-b07d-44f21538b211 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.860377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c57d25-1769-472a-b07d-44f21538b211 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.860648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c57d25-1769-472a-b07d-44f21538b211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:17.940655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded5626-bd2d-49d4-9fa3-a6f9417c75aa admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:17.942908 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-17c57d25-1769-472a-b07d-44f21538b211 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:17.944891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-17c57d25-1769-472a-b07d-44f21538b211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:17.966774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf1f72-873f-4b7c-bdc1-7986f71bde14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:17.967688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf1f72-873f-4b7c-bdc1-7986f71bde14 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.968290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf1f72-873f-4b7c-bdc1-7986f71bde14 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.969506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf1f72-873f-4b7c-bdc1-7986f71bde14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:17.972295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2ddf1f72-873f-4b7c-bdc1-7986f71bde14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:17.974764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 845/1704] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:17.977686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8fc12-f8f3-43eb-9615-d0ed6742a57d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:17.978844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8fc12-f8f3-43eb-9615-d0ed6742a57d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:17.981123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8fc12-f8f3-43eb-9615-d0ed6742a57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:18.054943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:18.140449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-a0d8fc12-f8f3-43eb-9615-d0ed6742a57d None None] MFA Rules not processed for user `e5582d10ebef459084fe287c18568f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:18.142033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a0d8fc12-f8f3-43eb-9615-d0ed6742a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:18.146689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:18.173744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf1f72-873f-4b7c-bdc1-7986f71bde14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:18.175378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf1f72-873f-4b7c-bdc1-7986f71bde14 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:18.220392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 860/1705] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:18.221308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf1f72-873f-4b7c-bdc1-7986f71bde14 service nova] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:18.284208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf1f72-873f-4b7c-bdc1-7986f71bde14 service nova] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:18.296662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 846/1706] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:18.317634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a677c-9092-44c9-926a-ea7b24909939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:18.318926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a677c-9092-44c9-926a-ea7b24909939 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:18.319346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a677c-9092-44c9-926a-ea7b24909939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:18.319979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a677c-9092-44c9-926a-ea7b24909939 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:18.320879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a677c-9092-44c9-926a-ea7b24909939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:18.321328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a677c-9092-44c9-926a-ea7b24909939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:18.324864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a677c-9092-44c9-926a-ea7b24909939 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:18.350961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 861/1707] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:52:18.355712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8fc12-f8f3-43eb-9615-d0ed6742a57d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:18.375168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8fc12-f8f3-43eb-9615-d0ed6742a57d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:18.387736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e5850-0932-4f3f-8601-69151105bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:18.389342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e5850-0932-4f3f-8601-69151105bcaa admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:18.390097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e5850-0932-4f3f-8601-69151105bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:18.391188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e5850-0932-4f3f-8601-69151105bcaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:18.402381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e5850-0932-4f3f-8601-69151105bcaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:18.411823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e5850-0932-4f3f-8601-69151105bcaa admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:18.597509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 847/1708] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:18 2022] POST /identity/v3/projects => generated 375 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:18.603251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a677c-9092-44c9-926a-ea7b24909939 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:18.627207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a677c-9092-44c9-926a-ea7b24909939 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:18.802485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd588e2-0693-4d4c-b27a-e4f82807faa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:18.803233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd588e2-0693-4d4c-b27a-e4f82807faa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:18.803233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd588e2-0693-4d4c-b27a-e4f82807faa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:18.803422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd588e2-0693-4d4c-b27a-e4f82807faa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:18.806224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd588e2-0693-4d4c-b27a-e4f82807faa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:18.812770 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd588e2-0693-4d4c-b27a-e4f82807faa9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:18.882839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39883ac-82f5-4f4c-b0f0-53f19f5ea9f1 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:18.883234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39883ac-82f5-4f4c-b0f0-53f19f5ea9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:18.883596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39883ac-82f5-4f4c-b0f0-53f19f5ea9f1 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:18.886141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 848/1709] 10.222.0.82 () {54 vars in 912 bytes} [Wed Aug 24 03:52:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:52:18.892266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108a2bc-5caf-47eb-adcc-f939ec48e77d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:18.892266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108a2bc-5caf-47eb-adcc-f939ec48e77d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:18.892266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108a2bc-5caf-47eb-adcc-f939ec48e77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:18.933177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 862/1710] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:18 2022] POST /identity/v3/users => generated 288 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:18.934471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd588e2-0693-4d4c-b27a-e4f82807faa9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:18.950645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd588e2-0693-4d4c-b27a-e4f82807faa9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:18.960425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35edd2-9a9a-4576-81e9-8ab2c641ca81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:18.963928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35edd2-9a9a-4576-81e9-8ab2c641ca81 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:18.963928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35edd2-9a9a-4576-81e9-8ab2c641ca81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:18.963928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35edd2-9a9a-4576-81e9-8ab2c641ca81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:18.967282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35edd2-9a9a-4576-81e9-8ab2c641ca81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:18.969745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35edd2-9a9a-4576-81e9-8ab2c641ca81 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:19.005487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7108a2bc-5caf-47eb-adcc-f939ec48e77d None None] MFA Rules not processed for user `f81cc77e85f84d358471dd9efab45cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:19.005487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7108a2bc-5caf-47eb-adcc-f939ec48e77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:19.012316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 863/1711] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:52:19.017659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35edd2-9a9a-4576-81e9-8ab2c641ca81 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:19.057870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35edd2-9a9a-4576-81e9-8ab2c641ca81 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:19.093841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb864b-b4d2-461e-83cd-571e1d758bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:19.095040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb864b-b4d2-461e-83cd-571e1d758bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:19.096757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb864b-b4d2-461e-83cd-571e1d758bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:19.097142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb864b-b4d2-461e-83cd-571e1d758bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:19.099984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb864b-b4d2-461e-83cd-571e1d758bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:19.103385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb864b-b4d2-461e-83cd-571e1d758bd5 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:19.147487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 864/1712] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:19.155422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 849/1713] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:52:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:19.157151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb864b-b4d2-461e-83cd-571e1d758bd5 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:19.174303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108a2bc-5caf-47eb-adcc-f939ec48e77d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:19.177067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7108a2bc-5caf-47eb-adcc-f939ec48e77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:19.205284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108a2bc-5caf-47eb-adcc-f939ec48e77d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:19.220828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb864b-b4d2-461e-83cd-571e1d758bd5 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:19.226716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:19.229516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5725be-1735-4030-8726-d7f377fea7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:19.231569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5725be-1735-4030-8726-d7f377fea7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:19.231855 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5725be-1735-4030-8726-d7f377fea7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:19.234628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5725be-1735-4030-8726-d7f377fea7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:19.242898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937dc3d-071f-4050-a905-3529fab0207e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:19.245916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937dc3d-071f-4050-a905-3529fab0207e admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:19.246408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937dc3d-071f-4050-a905-3529fab0207e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:19.246797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937dc3d-071f-4050-a905-3529fab0207e admin admin] PATH_INFO: `/v3/projects/0de750726f0948409b8ea54516621733/users/9fd83daa892d4af2a8d2fd79bd4beb6b/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:19.266734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5725be-1735-4030-8726-d7f377fea7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:19.273251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5725be-1735-4030-8726-d7f377fea7f5 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:19.316226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 850/1714] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:19.320778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5725be-1735-4030-8726-d7f377fea7f5 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:19.323527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937dc3d-071f-4050-a905-3529fab0207e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de750726f0948409b8ea54516621733, user_id=9fd83daa892d4af2a8d2fd79bd4beb6b, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:19.327479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5c5725be-1735-4030-8726-d7f377fea7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:19.333608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937dc3d-071f-4050-a905-3529fab0207e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:19.374969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5725be-1735-4030-8726-d7f377fea7f5 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:19.387682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:19.388364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:19.388654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:19.390391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] PATH_INFO: `/v3/projects/e04af3a5c44f4efd833ee8866d3e1fb4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:19.392944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e04af3a5c44f4efd833ee8866d3e1fb4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:19.398046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:19.481909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 865/1715] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:19 2022] PUT /identity/v3/projects/0de750726f0948409b8ea54516621733/users/9fd83daa892d4af2a8d2fd79bd4beb6b/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:19.493671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705918b-47c4-4cce-8d79-9b2d1002ffc9 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:19.494017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705918b-47c4-4cce-8d79-9b2d1002ffc9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:19.494356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705918b-47c4-4cce-8d79-9b2d1002ffc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:19.580224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5705918b-47c4-4cce-8d79-9b2d1002ffc9 None None] MFA Rules not processed for user `9fd83daa892d4af2a8d2fd79bd4beb6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:19.581490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5705918b-47c4-4cce-8d79-9b2d1002ffc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:19.693844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e04af3a5c44f4efd833ee8866d3e1fb4', 'request_id': 'req-84391428-0fc5-4993-8b11-b1fbcfa21df6'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:19.720118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 866/1716] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:19.721625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b06e3-6a66-449c-80d3-cea370856261 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:19.721923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b06e3-6a66-449c-80d3-cea370856261 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:19.729619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b06e3-6a66-449c-80d3-cea370856261 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:19.731498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 867/1717] 10.222.0.82 () {54 vars in 912 bytes} [Wed Aug 24 03:52:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:52:19.732374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b06e3-6a66-449c-80d3-cea370856261 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:19.736822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-306b06e3-6a66-449c-80d3-cea370856261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:19.837344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e04af3a5c44f4efd833ee8866d3e1fb4', 'request_id': 'req-84391428-0fc5-4993-8b11-b1fbcfa21df6'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:19.847337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e04af3a5c44f4efd833ee8866d3e1fb4', 'request_id': 'req-84391428-0fc5-4993-8b11-b1fbcfa21df6'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:19.866346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b06e3-6a66-449c-80d3-cea370856261 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:19.871835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d7cce-c756-4da0-b387-f6056651f839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:19.872681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d7cce-c756-4da0-b387-f6056651f839 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:19.872875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d7cce-c756-4da0-b387-f6056651f839 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:19.873162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d7cce-c756-4da0-b387-f6056651f839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:19.894110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] The token cache is being invalidate because project e04af3a5c44f4efd833ee8866d3e1fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:19.894475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:19.895928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-84391428-0fc5-4993-8b11-b1fbcfa21df6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:19.900232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 851/1718] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:19 2022] DELETE /identity/v3/projects/e04af3a5c44f4efd833ee8866d3e1fb4 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:19.910912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531cfcc-0134-422b-904f-1c6b945ae243 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:19.911587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531cfcc-0134-422b-904f-1c6b945ae243 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:19.911830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531cfcc-0134-422b-904f-1c6b945ae243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:19.960162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-5531cfcc-0134-422b-904f-1c6b945ae243 None None] MFA Rules not processed for user `f81cc77e85f84d358471dd9efab45cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:19.965385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5531cfcc-0134-422b-904f-1c6b945ae243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:20.008949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d7cce-c756-4da0-b387-f6056651f839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:20.010518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d7cce-c756-4da0-b387-f6056651f839 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:20.038268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 868/1719] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:20.038872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d7cce-c756-4da0-b387-f6056651f839 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:20.050728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-842d7cce-c756-4da0-b387-f6056651f839 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:20.102547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 852/1720] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:52:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:20.108995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531cfcc-0134-422b-904f-1c6b945ae243 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:20.113709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5531cfcc-0134-422b-904f-1c6b945ae243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:20.235210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:20.287659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d7cce-c756-4da0-b387-f6056651f839 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:20.315712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:20.316398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:20.316686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:20.316970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] PATH_INFO: `/v3/projects/9300ca7749774c998eeb2e3cfa01bcd1` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:20.324120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9300ca7749774c998eeb2e3cfa01bcd1)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:20.325372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531cfcc-0134-422b-904f-1c6b945ae243 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:20.326101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:20.342309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:20.343067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:20.345953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:20.345953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] PATH_INFO: `/v3/users/8abb1033dd1e4e0ba9d482a9d139c90d` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:20.346718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=8abb1033dd1e4e0ba9d482a9d139c90d)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:20.348960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:20.622932 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9300ca7749774c998eeb2e3cfa01bcd1', 'request_id': 'req-31b561b4-5a35-41f8-9147-6e389c6c30c7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.634622 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8abb1033dd1e4e0ba9d482a9d139c90d'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.637931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9300ca7749774c998eeb2e3cfa01bcd1', 'request_id': 'req-31b561b4-5a35-41f8-9147-6e389c6c30c7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.774549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8abb1033dd1e4e0ba9d482a9d139c90d'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.793794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8abb1033dd1e4e0ba9d482a9d139c90d'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.802821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8abb1033dd1e4e0ba9d482a9d139c90d'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.804062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8abb1033dd1e4e0ba9d482a9d139c90d'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.825583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 853/1721] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:20 2022] DELETE /identity/v3/users/8abb1033dd1e4e0ba9d482a9d139c90d => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:20.830075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:20.833279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:20.860255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9300ca7749774c998eeb2e3cfa01bcd1', 'request_id': 'req-31b561b4-5a35-41f8-9147-6e389c6c30c7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.887919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] The token cache is being invalidate because project 9300ca7749774c998eeb2e3cfa01bcd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:20.888579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.890489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:20.894554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 869/1722] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:20 2022] DELETE /identity/v3/projects/9300ca7749774c998eeb2e3cfa01bcd1 => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:20.894554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:20.907665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:20.969215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc5e0f-c4f3-4761-9cc0-9ec4afab92ab admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:20.980687 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65e59b-3025-4417-ad14-284f0cbfd6bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:20.981676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65e59b-3025-4417-ad14-284f0cbfd6bf service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:20.982096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65e59b-3025-4417-ad14-284f0cbfd6bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:20.983317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65e59b-3025-4417-ad14-284f0cbfd6bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:20.988744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ef65e59b-3025-4417-ad14-284f0cbfd6bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:21.024601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65e59b-3025-4417-ad14-284f0cbfd6bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:21.026898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65e59b-3025-4417-ad14-284f0cbfd6bf service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:21.039177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 854/1723] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:21.040336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db432b-5e30-43d3-bfe9-872c0e8ffc88 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:21.040628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db432b-5e30-43d3-bfe9-872c0e8ffc88 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:21.042043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db432b-5e30-43d3-bfe9-872c0e8ffc88 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:21.043800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 855/1724] 10.222.0.82 () {54 vars in 961 bytes} [Wed Aug 24 03:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:52:21.045378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82772abc-b12b-4bc9-bf56-a0d9ea0b809d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:21.045672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82772abc-b12b-4bc9-bf56-a0d9ea0b809d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:21.046267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82772abc-b12b-4bc9-bf56-a0d9ea0b809d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:21.076298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b561b4-5a35-41f8-9147-6e389c6c30c7 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:21.115979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-82772abc-b12b-4bc9-bf56-a0d9ea0b809d None None] MFA Rules not processed for user `c7e29563153a4ecbbe7447bed0324df7`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:21.120242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-82772abc-b12b-4bc9-bf56-a0d9ea0b809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:21.259228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7ef3c-260d-40a2-acef-04725c1ad346 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:21.260004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 856/1725] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:21.261877 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82772abc-b12b-4bc9-bf56-a0d9ea0b809d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:21.264507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-82772abc-b12b-4bc9-bf56-a0d9ea0b809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:21.267975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ef3c-260d-40a2-acef-04725c1ad346 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:21.268272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ef3c-260d-40a2-acef-04725c1ad346 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:21.268557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7ef3c-260d-40a2-acef-04725c1ad346 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:21.270670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-37d7ef3c-260d-40a2-acef-04725c1ad346 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:21.368164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ef3c-260d-40a2-acef-04725c1ad346 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:21.371452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7ef3c-260d-40a2-acef-04725c1ad346 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:21.399835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 870/1726] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:21.406325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cb60d-07ae-4496-9abf-a96e5ea198d4 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:21.406992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cb60d-07ae-4496-9abf-a96e5ea198d4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:21.407305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cb60d-07ae-4496-9abf-a96e5ea198d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:21.424825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82772abc-b12b-4bc9-bf56-a0d9ea0b809d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:21.449711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:21.450600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:21.450895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:21.451171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] PATH_INFO: `/v3/users/08173118774341f2abce9abaad176220` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:21.453733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08173118774341f2abce9abaad176220)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:21.456083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:21.460963 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-293cb60d-07ae-4496-9abf-a96e5ea198d4 None None] MFA Rules not processed for user `cff23ae056ff452588cd0b6520e96962`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:21.466081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-293cb60d-07ae-4496-9abf-a96e5ea198d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:21.586255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:21.604418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 871/1727] 10.222.0.82 () {58 vars in 1058 bytes} [Wed Aug 24 03:52:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:21.614116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cb60d-07ae-4496-9abf-a96e5ea198d4 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:21.645698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cb60d-07ae-4496-9abf-a96e5ea198d4 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:21.671723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882e8e9-dc3e-4013-bfcb-8e812602f78f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:21.673478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882e8e9-dc3e-4013-bfcb-8e812602f78f service placement] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:21.673478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882e8e9-dc3e-4013-bfcb-8e812602f78f service placement] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:21.673478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882e8e9-dc3e-4013-bfcb-8e812602f78f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:21.675524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c882e8e9-dc3e-4013-bfcb-8e812602f78f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:21.721281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882e8e9-dc3e-4013-bfcb-8e812602f78f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:21.723522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882e8e9-dc3e-4013-bfcb-8e812602f78f service placement] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:21.743158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08173118774341f2abce9abaad176220'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:21.754415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 872/1728] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:52:21.837266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08173118774341f2abce9abaad176220'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:21.855605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08173118774341f2abce9abaad176220'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:21.871126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08173118774341f2abce9abaad176220'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:21.871955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08173118774341f2abce9abaad176220'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:21.886634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 857/1729] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:21 2022] DELETE /identity/v3/users/08173118774341f2abce9abaad176220 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:21.926992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882e8e9-dc3e-4013-bfcb-8e812602f78f service placement] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:21.946854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882e8e9-dc3e-4013-bfcb-8e812602f78f service placement] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:22.093046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c63737-6b9c-46c6-a0b6-26853b0e96cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:22.093046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c63737-6b9c-46c6-a0b6-26853b0e96cf service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:22.093046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c63737-6b9c-46c6-a0b6-26853b0e96cf service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:22.093315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c63737-6b9c-46c6-a0b6-26853b0e96cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:22.095376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-88c63737-6b9c-46c6-a0b6-26853b0e96cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:22.179390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c63737-6b9c-46c6-a0b6-26853b0e96cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:22.181041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c63737-6b9c-46c6-a0b6-26853b0e96cf service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:22.195034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:22.198654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:22.200038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 873/1730] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:22.402625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b65a1d-e77f-47e0-a38d-c47f7aa7e6a4 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:22.411144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:22.411817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:22.412101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:22.412379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] PATH_INFO: `/v3/projects/c5246b48c42a47e981a37316348f0558` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:22.413829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5246b48c42a47e981a37316348f0558)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:22.423882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:22.608829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9048a63-4e3a-4f8a-8247-575f492e4a48 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:22.609119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9048a63-4e3a-4f8a-8247-575f492e4a48 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:22.609397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9048a63-4e3a-4f8a-8247-575f492e4a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:22.701101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-f9048a63-4e3a-4f8a-8247-575f492e4a48 None None] MFA Rules not processed for user `547bce0d8133462c95bcb6f272776a27`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:22.702866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f9048a63-4e3a-4f8a-8247-575f492e4a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:22.727399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5246b48c42a47e981a37316348f0558', 'request_id': 'req-647667c3-0c08-4df4-a13c-fa40917b8cd3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:22.779980 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:22.808290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5246b48c42a47e981a37316348f0558', 'request_id': 'req-647667c3-0c08-4df4-a13c-fa40917b8cd3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:22.820245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5246b48c42a47e981a37316348f0558', 'request_id': 'req-647667c3-0c08-4df4-a13c-fa40917b8cd3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:22.854706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] The token cache is being invalidate because project c5246b48c42a47e981a37316348f0558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:22.855162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:22.857262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-647667c3-0c08-4df4-a13c-fa40917b8cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:22.860171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 858/1731] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:22 2022] DELETE /identity/v3/projects/c5246b48c42a47e981a37316348f0558 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:22.865224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7b6bf-0484-49df-9b35-b8c73a62adbc None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:22.865519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7b6bf-0484-49df-9b35-b8c73a62adbc None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:22.865848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7b6bf-0484-49df-9b35-b8c73a62adbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:22.958716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 874/1732] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:22.966629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d8d7b6bf-0484-49df-9b35-b8c73a62adbc None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:22.970063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d8d7b6bf-0484-49df-9b35-b8c73a62adbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:23.302983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 859/1733] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:23.312061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccbc95-f0b7-4245-9855-86e06b5c894b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:23.312280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccbc95-f0b7-4245-9855-86e06b5c894b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:23.312565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccbc95-f0b7-4245-9855-86e06b5c894b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:23.402698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-61ccbc95-f0b7-4245-9855-86e06b5c894b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:23.403978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-61ccbc95-f0b7-4245-9855-86e06b5c894b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:23.437161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7b6bf-0484-49df-9b35-b8c73a62adbc None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:23.444733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 875/1734] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:23.451516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7b6bf-0484-49df-9b35-b8c73a62adbc None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:23.467557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32376d02-f7eb-4ac6-8901-a4f449ce52c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:23.468430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32376d02-f7eb-4ac6-8901-a4f449ce52c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:23.468874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32376d02-f7eb-4ac6-8901-a4f449ce52c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:23.469275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32376d02-f7eb-4ac6-8901-a4f449ce52c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:23.470096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32376d02-f7eb-4ac6-8901-a4f449ce52c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:23.470561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32376d02-f7eb-4ac6-8901-a4f449ce52c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:23.472325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32376d02-f7eb-4ac6-8901-a4f449ce52c8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:23.507362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 860/1735] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:52:23.524277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccbc95-f0b7-4245-9855-86e06b5c894b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:23.534572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccbc95-f0b7-4245-9855-86e06b5c894b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:23.540093 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0238fd-900e-4480-a43a-f383b5548254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:23.541306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0238fd-900e-4480-a43a-f383b5548254 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:23.541586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0238fd-900e-4480-a43a-f383b5548254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:23.541862 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0238fd-900e-4480-a43a-f383b5548254 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:23.542498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0238fd-900e-4480-a43a-f383b5548254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:23.544452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0238fd-900e-4480-a43a-f383b5548254 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:23.705012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32376d02-f7eb-4ac6-8901-a4f449ce52c8 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:23.706525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 876/1736] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:23 2022] POST /identity/v3/projects => generated 353 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:23.721750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32376d02-f7eb-4ac6-8901-a4f449ce52c8 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:23.854792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47aefe-2841-454a-a48d-a5c767001e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:23.855765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47aefe-2841-454a-a48d-a5c767001e7a admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:23.856372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47aefe-2841-454a-a48d-a5c767001e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:23.856836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47aefe-2841-454a-a48d-a5c767001e7a admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:23.858581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47aefe-2841-454a-a48d-a5c767001e7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:23.867186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47aefe-2841-454a-a48d-a5c767001e7a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:24.059491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 861/1737] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:23 2022] POST /identity/v3/users => generated 277 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:24.063869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0238fd-900e-4480-a43a-f383b5548254 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:24.094327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0238fd-900e-4480-a43a-f383b5548254 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:24.118061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5135d-fe95-4489-8d64-1f62b8ea0a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:24.119920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5135d-fe95-4489-8d64-1f62b8ea0a06 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:24.121844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5135d-fe95-4489-8d64-1f62b8ea0a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:24.128268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5135d-fe95-4489-8d64-1f62b8ea0a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:24.129420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5135d-fe95-4489-8d64-1f62b8ea0a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:24.139870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5135d-fe95-4489-8d64-1f62b8ea0a06 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:24.202208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 877/1738] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:24.205671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47aefe-2841-454a-a48d-a5c767001e7a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:24.229822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47aefe-2841-454a-a48d-a5c767001e7a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:24.265144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a750fa7-82d5-4180-9f50-0e15512d2575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:24.266054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a750fa7-82d5-4180-9f50-0e15512d2575 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:24.266587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a750fa7-82d5-4180-9f50-0e15512d2575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:24.267103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a750fa7-82d5-4180-9f50-0e15512d2575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:24.269428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a750fa7-82d5-4180-9f50-0e15512d2575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:24.275477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a750fa7-82d5-4180-9f50-0e15512d2575 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:24.378106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 862/1739] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:24.379856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5135d-fe95-4489-8d64-1f62b8ea0a06 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:24.396900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5135d-fe95-4489-8d64-1f62b8ea0a06 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:24.427004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e701f3c-2532-4152-9613-2c124f50c488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:24.448816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e701f3c-2532-4152-9613-2c124f50c488 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:24.449281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e701f3c-2532-4152-9613-2c124f50c488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:24.449567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e701f3c-2532-4152-9613-2c124f50c488 admin admin] PATH_INFO: `/v3/projects/00e324df62b84929b831e49543331b8b/users/3f81d93f3c4e4cf7a9d78c1c790567bb/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:24.539724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:24.551428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e701f3c-2532-4152-9613-2c124f50c488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e324df62b84929b831e49543331b8b, user_id=3f81d93f3c4e4cf7a9d78c1c790567bb, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:24.568560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e701f3c-2532-4152-9613-2c124f50c488 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:24.694299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 878/1740] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:24 2022] PUT /identity/v3/projects/00e324df62b84929b831e49543331b8b/users/3f81d93f3c4e4cf7a9d78c1c790567bb/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:24.710429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bf7df-1f6e-4b8b-b6d2-aa0c603a2557 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:24.710966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bf7df-1f6e-4b8b-b6d2-aa0c603a2557 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:24.710966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bf7df-1f6e-4b8b-b6d2-aa0c603a2557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:24.774327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e701f3c-2532-4152-9613-2c124f50c488 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:24.777085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4e701f3c-2532-4152-9613-2c124f50c488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:24.824393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b88bf7df-1f6e-4b8b-b6d2-aa0c603a2557 None None] MFA Rules not processed for user `3f81d93f3c4e4cf7a9d78c1c790567bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:24.826650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b88bf7df-1f6e-4b8b-b6d2-aa0c603a2557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:24.992816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 863/1741] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:24.994372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bf7df-1f6e-4b8b-b6d2-aa0c603a2557 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:25.011863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e701f3c-2532-4152-9613-2c124f50c488 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:25.031154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bf7df-1f6e-4b8b-b6d2-aa0c603a2557 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:25.058947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:25.059930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:25.063108 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:25.063494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] PATH_INFO: `/v3/projects/d78f36597b2d4497a94b7e85a38c6a40` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:25.074557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d78f36597b2d4497a94b7e85a38c6a40)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:25.084787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:25.085747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75963b9-a4e6-4764-890f-696c11b351fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:25.089781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75963b9-a4e6-4764-890f-696c11b351fb admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:25.092266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75963b9-a4e6-4764-890f-696c11b351fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:25.093915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75963b9-a4e6-4764-890f-696c11b351fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:25.094607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75963b9-a4e6-4764-890f-696c11b351fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:25.096578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75963b9-a4e6-4764-890f-696c11b351fb admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:25.333852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 864/1742] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:24 2022] POST /identity/v3/projects => generated 357 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:25.336692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d70a0b-0e22-495f-9164-c70c873b710a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:25.338245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d70a0b-0e22-495f-9164-c70c873b710a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:25.343290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d70a0b-0e22-495f-9164-c70c873b710a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:25.363209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd78f36597b2d4497a94b7e85a38c6a40', 'request_id': 'req-e24c297d-61f3-4f76-a30a-8baa4b86936b'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:25.379688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd78f36597b2d4497a94b7e85a38c6a40', 'request_id': 'req-e24c297d-61f3-4f76-a30a-8baa4b86936b'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:25.390022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e3d70a0b-0e22-495f-9164-c70c873b710a None None] MFA Rules not processed for user `87a0f4d8b8c341ed85618e70adb2def4`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:25.390022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e3d70a0b-0e22-495f-9164-c70c873b710a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:25.512412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd78f36597b2d4497a94b7e85a38c6a40', 'request_id': 'req-e24c297d-61f3-4f76-a30a-8baa4b86936b'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:25.537360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 865/1743] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:25.538713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d70a0b-0e22-495f-9164-c70c873b710a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:25.558799 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] The token cache is being invalidate because project d78f36597b2d4497a94b7e85a38c6a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:25.565519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:25.567194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e24c297d-61f3-4f76-a30a-8baa4b86936b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:25.569788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 879/1744] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:24 2022] DELETE /identity/v3/projects/d78f36597b2d4497a94b7e85a38c6a40 => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:25.574920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ccff-c2cc-4240-9229-b368ff521ed3 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:25.575534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ccff-c2cc-4240-9229-b368ff521ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:25.576018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ccff-c2cc-4240-9229-b368ff521ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:25.633505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5293ccff-c2cc-4240-9229-b368ff521ed3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:25.635612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5293ccff-c2cc-4240-9229-b368ff521ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:25.809305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d70a0b-0e22-495f-9164-c70c873b710a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:25.827119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a4ef-999c-4fb8-b343-dd4df23c159d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:25.827853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a4ef-999c-4fb8-b343-dd4df23c159d admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:25.828138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a4ef-999c-4fb8-b343-dd4df23c159d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:25.828577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a4ef-999c-4fb8-b343-dd4df23c159d admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:25.829390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033a4ef-999c-4fb8-b343-dd4df23c159d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:25.831746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033a4ef-999c-4fb8-b343-dd4df23c159d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:25.846055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 880/1745] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:25.854323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f45f36-1f49-43da-bf62-0f96229ce0c7 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:25.855459 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f45f36-1f49-43da-bf62-0f96229ce0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:25.855975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f45f36-1f49-43da-bf62-0f96229ce0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:25.902342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a1f45f36-1f49-43da-bf62-0f96229ce0c7 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:25.902342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a1f45f36-1f49-43da-bf62-0f96229ce0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:25.923294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 881/1746] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:25.927294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f45f36-1f49-43da-bf62-0f96229ce0c7 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:25.949789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f45f36-1f49-43da-bf62-0f96229ce0c7 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:25.963019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06723a-a38e-4853-8133-d6abf5cab3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:25.965186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 866/1747] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:25 2022] POST /identity/v3/users => generated 285 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:25.965305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06723a-a38e-4853-8133-d6abf5cab3eb admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:25.965305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06723a-a38e-4853-8133-d6abf5cab3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:25.965305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06723a-a38e-4853-8133-d6abf5cab3eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:25.966603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06723a-a38e-4853-8133-d6abf5cab3eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:25.967036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06723a-a38e-4853-8133-d6abf5cab3eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:25.967484 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a4ef-999c-4fb8-b343-dd4df23c159d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:25.974683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06723a-a38e-4853-8133-d6abf5cab3eb admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:25.983475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7033a4ef-999c-4fb8-b343-dd4df23c159d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:26.009466 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a4ef-999c-4fb8-b343-dd4df23c159d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:26.013283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 882/1748] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:52:26.017703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06723a-a38e-4853-8133-d6abf5cab3eb admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:26.035610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e957f-1fc8-4cda-9d21-fd89dea16e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:26.036256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e957f-1fc8-4cda-9d21-fd89dea16e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:26.036542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e957f-1fc8-4cda-9d21-fd89dea16e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:26.046037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e957f-1fc8-4cda-9d21-fd89dea16e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:26.046037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e957f-1fc8-4cda-9d21-fd89dea16e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:26.046822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06723a-a38e-4853-8133-d6abf5cab3eb admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:26.046919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e957f-1fc8-4cda-9d21-fd89dea16e16 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:26.056217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384a852-bc87-4d61-bc1e-f6cb5a6e5d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:26.057232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a852-bc87-4d61-bc1e-f6cb5a6e5d16 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:26.057727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a852-bc87-4d61-bc1e-f6cb5a6e5d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:26.058212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a852-bc87-4d61-bc1e-f6cb5a6e5d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:26.059627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384a852-bc87-4d61-bc1e-f6cb5a6e5d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:26.061818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384a852-bc87-4d61-bc1e-f6cb5a6e5d16 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:26.093616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 867/1749] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:26.094656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e957f-1fc8-4cda-9d21-fd89dea16e16 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:26.110144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e957f-1fc8-4cda-9d21-fd89dea16e16 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:26.127115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de332b1-514a-420b-8bd7-e211cf737ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:26.127817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de332b1-514a-420b-8bd7-e211cf737ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:26.128196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de332b1-514a-420b-8bd7-e211cf737ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:26.128511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de332b1-514a-420b-8bd7-e211cf737ee9 admin admin] PATH_INFO: `/v3/projects/24c63591df1a473e9a877f2f8a9d48a9/users/08b46aa98c5a4f37824026c4fa32b062/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:26.161524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de332b1-514a-420b-8bd7-e211cf737ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c63591df1a473e9a877f2f8a9d48a9, user_id=08b46aa98c5a4f37824026c4fa32b062, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:26.164281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de332b1-514a-420b-8bd7-e211cf737ee9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:26.299515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 883/1750] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:26 2022] POST /identity/v3/projects => generated 399 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:26.303419 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384a852-bc87-4d61-bc1e-f6cb5a6e5d16 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:26.306463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 868/1751] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:26 2022] PUT /identity/v3/projects/24c63591df1a473e9a877f2f8a9d48a9/users/08b46aa98c5a4f37824026c4fa32b062/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:26.310087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de332b1-514a-420b-8bd7-e211cf737ee9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:26.322575 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:26.467258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de332b1-514a-420b-8bd7-e211cf737ee9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:26.471399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384a852-bc87-4d61-bc1e-f6cb5a6e5d16 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:26.480170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f4e1-dabd-47a5-a481-9513da9bc71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:26.481467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f4e1-dabd-47a5-a481-9513da9bc71e admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:26.484737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f4e1-dabd-47a5-a481-9513da9bc71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:26.484737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f4e1-dabd-47a5-a481-9513da9bc71e admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:26.484737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28f4e1-dabd-47a5-a481-9513da9bc71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:26.486004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28f4e1-dabd-47a5-a481-9513da9bc71e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:26.486863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4ead8-56d8-4ca4-b09b-9549d7605579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:26.487674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4ead8-56d8-4ca4-b09b-9549d7605579 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:26.488101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4ead8-56d8-4ca4-b09b-9549d7605579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:26.488676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4ead8-56d8-4ca4-b09b-9549d7605579 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:26.492078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4ead8-56d8-4ca4-b09b-9549d7605579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:26.496408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4ead8-56d8-4ca4-b09b-9549d7605579 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:26.543549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 869/1752] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:26.557823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f4e1-dabd-47a5-a481-9513da9bc71e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:26.564067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6d28f4e1-dabd-47a5-a481-9513da9bc71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:26.610407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f4e1-dabd-47a5-a481-9513da9bc71e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:26.619530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:26.620449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:26.620881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:26.621311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] PATH_INFO: `/v3/users/9fd83daa892d4af2a8d2fd79bd4beb6b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:26.623652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fd83daa892d4af2a8d2fd79bd4beb6b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:26.625864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:26.735223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 884/1753] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:26 2022] POST /identity/v3/users => generated 300 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:26.736330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4ead8-56d8-4ca4-b09b-9549d7605579 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:26.753631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4ead8-56d8-4ca4-b09b-9549d7605579 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:26.762698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831149-cef7-4154-9c4c-39e0b9c1056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:26.766152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831149-cef7-4154-9c4c-39e0b9c1056c admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:26.766152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831149-cef7-4154-9c4c-39e0b9c1056c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:26.766152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831149-cef7-4154-9c4c-39e0b9c1056c admin admin] PATH_INFO: `/v3/projects/24c63591df1a473e9a877f2f8a9d48a9/users/08b46aa98c5a4f37824026c4fa32b062/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:26.767553 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5831149-cef7-4154-9c4c-39e0b9c1056c admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c63591df1a473e9a877f2f8a9d48a9, user_id=08b46aa98c5a4f37824026c4fa32b062, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:26.771570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5831149-cef7-4154-9c4c-39e0b9c1056c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:26.787229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 885/1754] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:26 2022] PUT /identity/v3/projects/24c63591df1a473e9a877f2f8a9d48a9/users/08b46aa98c5a4f37824026c4fa32b062/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:26.793572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831149-cef7-4154-9c4c-39e0b9c1056c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:26.937414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fd83daa892d4af2a8d2fd79bd4beb6b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:26.953037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831149-cef7-4154-9c4c-39e0b9c1056c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:26.968190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a986b1-6e88-42d3-9f7e-f07f34de2fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:26.968843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a986b1-6e88-42d3-9f7e-f07f34de2fec admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:26.969134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a986b1-6e88-42d3-9f7e-f07f34de2fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:26.969415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a986b1-6e88-42d3-9f7e-f07f34de2fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:26.970107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a986b1-6e88-42d3-9f7e-f07f34de2fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:26.981919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a986b1-6e88-42d3-9f7e-f07f34de2fec admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:27.017755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 886/1755] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:27.019328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe34e8-c8f2-4411-8787-9d35137a5d40 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:27.019647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe34e8-c8f2-4411-8787-9d35137a5d40 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:27.019931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe34e8-c8f2-4411-8787-9d35137a5d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:27.093099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fd83daa892d4af2a8d2fd79bd4beb6b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:27.119661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fd83daa892d4af2a8d2fd79bd4beb6b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:27.134571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-78fe34e8-c8f2-4411-8787-9d35137a5d40 None None] MFA Rules not processed for user `08b46aa98c5a4f37824026c4fa32b062`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:27.136008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-78fe34e8-c8f2-4411-8787-9d35137a5d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:27.160396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fd83daa892d4af2a8d2fd79bd4beb6b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:27.161784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fd83daa892d4af2a8d2fd79bd4beb6b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:27.197748 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 870/1756] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:26 2022] DELETE /identity/v3/users/9fd83daa892d4af2a8d2fd79bd4beb6b => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:27.200794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:27.221595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:27.257890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:27.328137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 887/1757] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:27.340112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f13c9-3070-4369-9992-79ef5a8320b6 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:27.341424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f13c9-3070-4369-9992-79ef5a8320b6 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:27.341851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f13c9-3070-4369-9992-79ef5a8320b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:27.413096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-938f13c9-3070-4369-9992-79ef5a8320b6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:27.420888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-938f13c9-3070-4369-9992-79ef5a8320b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:27.503676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:27.515436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921d166-d645-49d8-94a1-e07b18ebd6f0 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:27.546395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6684a29-252f-4054-9683-e318da30b79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:27.548244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6684a29-252f-4054-9683-e318da30b79b admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:27.548741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6684a29-252f-4054-9683-e318da30b79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:27.549245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6684a29-252f-4054-9683-e318da30b79b admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:27.550357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6684a29-252f-4054-9683-e318da30b79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:27.553095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6684a29-252f-4054-9683-e318da30b79b admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:27.577728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 871/1758] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:27.578875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6684a29-252f-4054-9683-e318da30b79b admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:27.582273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d6684a29-252f-4054-9683-e318da30b79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:27.589061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 888/1759] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:27.589061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f13c9-3070-4369-9992-79ef5a8320b6 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:27.604470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f13c9-3070-4369-9992-79ef5a8320b6 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:27.622150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9570484-2104-4e9b-be93-e249a9e2c67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:27.623371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9570484-2104-4e9b-be93-e249a9e2c67c admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:27.623958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9570484-2104-4e9b-be93-e249a9e2c67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:27.624407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9570484-2104-4e9b-be93-e249a9e2c67c admin admin] PATH_INFO: `/v3/projects/2ec9afcce98943b8aa21dfefd305d4a8/users/24e4a337135a483ebfa25813913e376c/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:27.689487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6684a29-252f-4054-9683-e318da30b79b admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:27.696070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55afe27-dba7-4b9a-8622-cca6ac2abb16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:27.697318 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9570484-2104-4e9b-be93-e249a9e2c67c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec9afcce98943b8aa21dfefd305d4a8, user_id=24e4a337135a483ebfa25813913e376c, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:27.698078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55afe27-dba7-4b9a-8622-cca6ac2abb16 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:27.698569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55afe27-dba7-4b9a-8622-cca6ac2abb16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:27.699015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55afe27-dba7-4b9a-8622-cca6ac2abb16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:27.700123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9570484-2104-4e9b-be93-e249a9e2c67c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:27.798708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55afe27-dba7-4b9a-8622-cca6ac2abb16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:27.800539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55afe27-dba7-4b9a-8622-cca6ac2abb16 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:27.812369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 872/1760] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:27.813814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55afe27-dba7-4b9a-8622-cca6ac2abb16 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:27.815637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d55afe27-dba7-4b9a-8622-cca6ac2abb16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:27.849388 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 889/1761] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:27 2022] PUT /identity/v3/projects/2ec9afcce98943b8aa21dfefd305d4a8/users/24e4a337135a483ebfa25813913e376c/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:27.864832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b51f3-8232-4dc4-bb84-ec0f571d82c0 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:27.865215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b51f3-8232-4dc4-bb84-ec0f571d82c0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:27.865491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b51f3-8232-4dc4-bb84-ec0f571d82c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:27.891963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55afe27-dba7-4b9a-8622-cca6ac2abb16 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:27.936297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a93b51f3-8232-4dc4-bb84-ec0f571d82c0 None None] MFA Rules not processed for user `24e4a337135a483ebfa25813913e376c`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:27.937589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a93b51f3-8232-4dc4-bb84-ec0f571d82c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:28.027234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53146fef-bf53-4bb8-b131-c7a26528158e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:28.028093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53146fef-bf53-4bb8-b131-c7a26528158e service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:28.029117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53146fef-bf53-4bb8-b131-c7a26528158e service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:28.029556 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53146fef-bf53-4bb8-b131-c7a26528158e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:28.151983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 890/1762] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:28.155153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b51f3-8232-4dc4-bb84-ec0f571d82c0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:28.177941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53146fef-bf53-4bb8-b131-c7a26528158e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:28.178943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53146fef-bf53-4bb8-b131-c7a26528158e service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:28.212308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 873/1763] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:28.276779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b51f3-8232-4dc4-bb84-ec0f571d82c0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:28.279101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53146fef-bf53-4bb8-b131-c7a26528158e service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:28.282540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a6f13-1edf-499d-9c13-39c06f2156d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:28.283444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a6f13-1edf-499d-9c13-39c06f2156d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:28.283793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a6f13-1edf-499d-9c13-39c06f2156d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:28.284158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a6f13-1edf-499d-9c13-39c06f2156d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:28.286087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a6f13-1edf-499d-9c13-39c06f2156d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:28.288558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a6f13-1edf-499d-9c13-39c06f2156d8 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:28.429026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53146fef-bf53-4bb8-b131-c7a26528158e service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:28.436358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e436ea-242d-49c2-8daf-5a33cb57e6f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:28.437426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e436ea-242d-49c2-8daf-5a33cb57e6f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:28.437862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e436ea-242d-49c2-8daf-5a33cb57e6f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:28.438469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e436ea-242d-49c2-8daf-5a33cb57e6f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:28.441163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d8e436ea-242d-49c2-8daf-5a33cb57e6f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:28.453319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 891/1764] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:28 2022] POST /identity/v3/projects => generated 401 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:28.473023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:28.501693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a6f13-1edf-499d-9c13-39c06f2156d8 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:28.563086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e436ea-242d-49c2-8daf-5a33cb57e6f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:28.564785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e436ea-242d-49c2-8daf-5a33cb57e6f2 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:28.582039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 874/1765] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:28 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:28.663224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a6f13-1edf-499d-9c13-39c06f2156d8 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:28.685895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4a117-54d4-404f-80f8-e21e8f1052b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:28.685895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4a117-54d4-404f-80f8-e21e8f1052b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:28.685895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4a117-54d4-404f-80f8-e21e8f1052b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:28.685895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4a117-54d4-404f-80f8-e21e8f1052b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:28.686397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4a117-54d4-404f-80f8-e21e8f1052b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:28.686397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4a117-54d4-404f-80f8-e21e8f1052b8 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:28.798073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 892/1766] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:28 2022] POST /identity/v3/users => generated 307 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:28.802963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e436ea-242d-49c2-8daf-5a33cb57e6f2 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:28.829210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e436ea-242d-49c2-8daf-5a33cb57e6f2 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:28.842348 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b6053-88bb-4ea5-9435-4fa08fa663dd None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:28.842853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b6053-88bb-4ea5-9435-4fa08fa663dd None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:28.843313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b6053-88bb-4ea5-9435-4fa08fa663dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:28.877439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571814b0-2cd4-4b8f-8399-587a1267a170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:28.919012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571814b0-2cd4-4b8f-8399-587a1267a170 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:28.930309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571814b0-2cd4-4b8f-8399-587a1267a170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:28.930309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571814b0-2cd4-4b8f-8399-587a1267a170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:28.932092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571814b0-2cd4-4b8f-8399-587a1267a170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:28.934729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571814b0-2cd4-4b8f-8399-587a1267a170 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:28.960004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 875/1767] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:28.960989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571814b0-2cd4-4b8f-8399-587a1267a170 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:28.989910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571814b0-2cd4-4b8f-8399-587a1267a170 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:28.995944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a7570-6763-47dd-8aa4-7e9e60b33cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:28.996997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a7570-6763-47dd-8aa4-7e9e60b33cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:28.997363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a7570-6763-47dd-8aa4-7e9e60b33cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:28.997593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a7570-6763-47dd-8aa4-7e9e60b33cb8 admin admin] PATH_INFO: `/v3/projects/644fc16140324c7f89dc35f016cd5add/users/2a63adf7d5a74478bdfcfe0b4ce53fb9/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:29.028289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-f79b6053-88bb-4ea5-9435-4fa08fa663dd None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:29.029778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f79b6053-88bb-4ea5-9435-4fa08fa663dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:29.035495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a7570-6763-47dd-8aa4-7e9e60b33cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=644fc16140324c7f89dc35f016cd5add, user_id=2a63adf7d5a74478bdfcfe0b4ce53fb9, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:29.046224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a7570-6763-47dd-8aa4-7e9e60b33cb8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:29.150324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 893/1768] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:52:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:29.151955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f4aca-9126-4c5e-a65c-7acf4ca5024a None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:29.152394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f4aca-9126-4c5e-a65c-7acf4ca5024a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:29.152776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f4aca-9126-4c5e-a65c-7acf4ca5024a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:29.194755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-136f4aca-9126-4c5e-a65c-7acf4ca5024a None None] MFA Rules not processed for user `c7e29563153a4ecbbe7447bed0324df7`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:29.196522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-136f4aca-9126-4c5e-a65c-7acf4ca5024a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:29.200696 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 876/1769] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:28 2022] PUT /identity/v3/projects/644fc16140324c7f89dc35f016cd5add/users/2a63adf7d5a74478bdfcfe0b4ce53fb9/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:29.201959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a7570-6763-47dd-8aa4-7e9e60b33cb8 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:29.321915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 894/1770] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:29.327825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a7570-6763-47dd-8aa4-7e9e60b33cb8 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:29.335022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c29ab-1367-4ab1-b63f-9a5faa365d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:29.335900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c29ab-1367-4ab1-b63f-9a5faa365d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:29.336469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c29ab-1367-4ab1-b63f-9a5faa365d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:29.336966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c29ab-1367-4ab1-b63f-9a5faa365d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:29.338004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c29ab-1367-4ab1-b63f-9a5faa365d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:29.339898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c29ab-1367-4ab1-b63f-9a5faa365d9d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:29.373775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 877/1771] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:29.381411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27011bc0-7f86-44d0-8211-ccd1bf25de83 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:29.382554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27011bc0-7f86-44d0-8211-ccd1bf25de83 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:29.382887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27011bc0-7f86-44d0-8211-ccd1bf25de83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:29.383381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c29ab-1367-4ab1-b63f-9a5faa365d9d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:29.395979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c29ab-1367-4ab1-b63f-9a5faa365d9d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:29.457733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-27011bc0-7f86-44d0-8211-ccd1bf25de83 None None] MFA Rules not processed for user `e5582d10ebef459084fe287c18568f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:29.460082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-27011bc0-7f86-44d0-8211-ccd1bf25de83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:29.461788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8436d-0cf4-4c2e-8e6c-b54e402c3ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:29.462546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8436d-0cf4-4c2e-8e6c-b54e402c3ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:29.462889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8436d-0cf4-4c2e-8e6c-b54e402c3ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:29.463174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8436d-0cf4-4c2e-8e6c-b54e402c3ecd admin admin] PATH_INFO: `/v3/projects/644fc16140324c7f89dc35f016cd5add/users/2a63adf7d5a74478bdfcfe0b4ce53fb9/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:29.472701 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8436d-0cf4-4c2e-8e6c-b54e402c3ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=644fc16140324c7f89dc35f016cd5add, user_id=2a63adf7d5a74478bdfcfe0b4ce53fb9, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:29.482147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8436d-0cf4-4c2e-8e6c-b54e402c3ecd admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:29.498733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 878/1772] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:29 2022] PUT /identity/v3/projects/644fc16140324c7f89dc35f016cd5add/users/2a63adf7d5a74478bdfcfe0b4ce53fb9/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:29.514279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4015181-1126-4f15-a043-a00d9d082a4f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:29.514609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4015181-1126-4f15-a043-a00d9d082a4f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:29.514890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4015181-1126-4f15-a043-a00d9d082a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:29.578271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-a4015181-1126-4f15-a043-a00d9d082a4f None None] MFA Rules not processed for user `2a63adf7d5a74478bdfcfe0b4ce53fb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:29.580118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a4015181-1126-4f15-a043-a00d9d082a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:29.644004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 895/1773] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:29.687621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 879/1774] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:29 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:29.701198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27011bc0-7f86-44d0-8211-ccd1bf25de83 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:29.778761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4015181-1126-4f15-a043-a00d9d082a4f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:29.780916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a4015181-1126-4f15-a043-a00d9d082a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:29.808145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27011bc0-7f86-44d0-8211-ccd1bf25de83 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:29.899972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeead01-45e9-4590-9e8e-5266d22076d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:29.900608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeead01-45e9-4590-9e8e-5266d22076d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:29.900896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeead01-45e9-4590-9e8e-5266d22076d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:29.901175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeead01-45e9-4590-9e8e-5266d22076d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:29.919871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeead01-45e9-4590-9e8e-5266d22076d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:29.921753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeead01-45e9-4590-9e8e-5266d22076d9 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:29.968864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 896/1775] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:29 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:29.969895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeead01-45e9-4590-9e8e-5266d22076d9 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:29.975290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-aeeead01-45e9-4590-9e8e-5266d22076d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:30.083926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:30.107413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4015181-1126-4f15-a043-a00d9d082a4f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:30.144612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce0980-509c-4d81-83e2-842fd9afee9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:30.145507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce0980-509c-4d81-83e2-842fd9afee9f service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:30.145796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce0980-509c-4d81-83e2-842fd9afee9f service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:30.147932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce0980-509c-4d81-83e2-842fd9afee9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:30.235995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeead01-45e9-4590-9e8e-5266d22076d9 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:30.254659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:30.257695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:30.258081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:30.258368 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] PATH_INFO: `/v3/projects/0de750726f0948409b8ea54516621733` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:30.265581 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de750726f0948409b8ea54516621733)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:30.268364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:30.345280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce0980-509c-4d81-83e2-842fd9afee9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:30.350806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce0980-509c-4d81-83e2-842fd9afee9f service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:30.370147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 880/1776] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:52:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:52:30.377034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce0980-509c-4d81-83e2-842fd9afee9f service swift] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:30.407690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce0980-509c-4d81-83e2-842fd9afee9f service swift] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:30.434294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd622a9b-bf8e-43e2-b498-6c042f973b73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:30.436178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd622a9b-bf8e-43e2-b498-6c042f973b73 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:30.437445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd622a9b-bf8e-43e2-b498-6c042f973b73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:30.438897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd622a9b-bf8e-43e2-b498-6c042f973b73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:30.517262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de750726f0948409b8ea54516621733', 'request_id': 'req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:30.568897 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de750726f0948409b8ea54516621733', 'request_id': 'req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:30.726740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de750726f0948409b8ea54516621733', 'request_id': 'req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:30.758413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] The token cache is being invalidate because project 0de750726f0948409b8ea54516621733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:30.758744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:30.769387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-08aeb602-ccd9-4cf5-9731-d96a0dcfe41f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:30.774606 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 897/1777] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:29 2022] DELETE /identity/v3/projects/0de750726f0948409b8ea54516621733 => generated 0 bytes in 797 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:30.780910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1002ec-25fc-470b-8bd1-123fe7b43014 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:30.782969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1002ec-25fc-470b-8bd1-123fe7b43014 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:30.784700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1002ec-25fc-470b-8bd1-123fe7b43014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:30.843086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd622a9b-bf8e-43e2-b498-6c042f973b73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:30.854116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd622a9b-bf8e-43e2-b498-6c042f973b73 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:30.912943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 881/1778] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:30.920319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-bb1002ec-25fc-470b-8bd1-123fe7b43014 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:30.926723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bb1002ec-25fc-470b-8bd1-123fe7b43014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:31.268408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 898/1779] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:31.277475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b015b-fe93-4f02-b22d-46de4c77b7dc None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:31.280850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b015b-fe93-4f02-b22d-46de4c77b7dc None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:31.280850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b015b-fe93-4f02-b22d-46de4c77b7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:31.352900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e54b015b-fe93-4f02-b22d-46de4c77b7dc None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:31.357873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e54b015b-fe93-4f02-b22d-46de4c77b7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:31.416205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 882/1780] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:31.420211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1002ec-25fc-470b-8bd1-123fe7b43014 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:31.436902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1002ec-25fc-470b-8bd1-123fe7b43014 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:31.456781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c076dba-b333-4fce-8b79-88a5e840de07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:31.457567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c076dba-b333-4fce-8b79-88a5e840de07 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:31.462641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c076dba-b333-4fce-8b79-88a5e840de07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:31.463077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c076dba-b333-4fce-8b79-88a5e840de07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:31.464038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c076dba-b333-4fce-8b79-88a5e840de07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:31.466657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c076dba-b333-4fce-8b79-88a5e840de07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:31.471843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c076dba-b333-4fce-8b79-88a5e840de07 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:31.546304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b015b-fe93-4f02-b22d-46de4c77b7dc None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:31.564999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 899/1781] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:52:31.565547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b015b-fe93-4f02-b22d-46de4c77b7dc None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:31.575111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c62a4-64de-49f8-9d71-4ae1dc6525fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:31.575464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c62a4-64de-49f8-9d71-4ae1dc6525fd admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:31.575767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c62a4-64de-49f8-9d71-4ae1dc6525fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:31.576186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c62a4-64de-49f8-9d71-4ae1dc6525fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:31.577256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c62a4-64de-49f8-9d71-4ae1dc6525fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:31.579690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c62a4-64de-49f8-9d71-4ae1dc6525fd admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:31.769706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:31.789949 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 883/1782] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:31 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:52:31.794719 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c076dba-b333-4fce-8b79-88a5e840de07 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:31.843270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c076dba-b333-4fce-8b79-88a5e840de07 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:31.884759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a91c0-8fa5-4619-8e4e-ab08aeb4482c None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:31.885070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a91c0-8fa5-4619-8e4e-ab08aeb4482c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:31.885310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a91c0-8fa5-4619-8e4e-ab08aeb4482c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:31.914638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:31.994161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-8f0a91c0-8fa5-4619-8e4e-ab08aeb4482c None None] MFA Rules not processed for user `f81cc77e85f84d358471dd9efab45cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:31.996274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8f0a91c0-8fa5-4619-8e4e-ab08aeb4482c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:32.051256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21d9c9-b8c8-47ab-84af-b7b5512ed4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:32.051905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21d9c9-b8c8-47ab-84af-b7b5512ed4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:32.068037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21d9c9-b8c8-47ab-84af-b7b5512ed4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:32.069134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21d9c9-b8c8-47ab-84af-b7b5512ed4b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:32.071081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21d9c9-b8c8-47ab-84af-b7b5512ed4b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:32.078828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21d9c9-b8c8-47ab-84af-b7b5512ed4b1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:32.100853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 884/1783] 10.222.0.83 () {58 vars in 1015 bytes} [Wed Aug 24 03:52:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:32.139088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a91c0-8fa5-4619-8e4e-ab08aeb4482c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:32.141381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8f0a91c0-8fa5-4619-8e4e-ab08aeb4482c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:32.190860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a91c0-8fa5-4619-8e4e-ab08aeb4482c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:32.201118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453ab55-da42-4275-902d-4ff1f10acda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:32.202262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453ab55-da42-4275-902d-4ff1f10acda0 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:32.202861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453ab55-da42-4275-902d-4ff1f10acda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:32.203567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453ab55-da42-4275-902d-4ff1f10acda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:32.218788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453ab55-da42-4275-902d-4ff1f10acda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:32.220472 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453ab55-da42-4275-902d-4ff1f10acda0 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:32.242386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 885/1784] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:32.289885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453ab55-da42-4275-902d-4ff1f10acda0 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:32.290735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 900/1785] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:31 2022] POST /identity/v3/users => generated 293 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:32.310266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453ab55-da42-4275-902d-4ff1f10acda0 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:32.341080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a1e8b-c8bf-43e8-b4b8-ec0a581e6fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:32.345332 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a1e8b-c8bf-43e8-b4b8-ec0a581e6fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:32.345626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a1e8b-c8bf-43e8-b4b8-ec0a581e6fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:32.347397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a1e8b-c8bf-43e8-b4b8-ec0a581e6fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:32.348103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a1e8b-c8bf-43e8-b4b8-ec0a581e6fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:32.356579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a1e8b-c8bf-43e8-b4b8-ec0a581e6fa6 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:32.400812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21d9c9-b8c8-47ab-84af-b7b5512ed4b1 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:32.425697 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 886/1786] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:32.435920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21d9c9-b8c8-47ab-84af-b7b5512ed4b1 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:32.462377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8f6ef-d60c-4cef-b890-64782421b52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:32.479166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8f6ef-d60c-4cef-b890-64782421b52b admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:32.479166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8f6ef-d60c-4cef-b890-64782421b52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:32.480772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8f6ef-d60c-4cef-b890-64782421b52b admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:32.482001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8f6ef-d60c-4cef-b890-64782421b52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:32.578643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8f6ef-d60c-4cef-b890-64782421b52b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:32.578643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 901/1787] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:52:32.582664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a1e8b-c8bf-43e8-b4b8-ec0a581e6fa6 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:32.595069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a1e8b-c8bf-43e8-b4b8-ec0a581e6fa6 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:32.607954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf787c7e-6ec1-4cf0-a624-af99ccd77eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:32.608686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf787c7e-6ec1-4cf0-a624-af99ccd77eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:32.608967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf787c7e-6ec1-4cf0-a624-af99ccd77eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:32.611862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf787c7e-6ec1-4cf0-a624-af99ccd77eb0 admin admin] PATH_INFO: `/v3/projects/3626d81cf984436aa570cdb0171723ad/users/b7cdfb59a7904cb793bf94af35d8c292/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:32.678559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf787c7e-6ec1-4cf0-a624-af99ccd77eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3626d81cf984436aa570cdb0171723ad, user_id=b7cdfb59a7904cb793bf94af35d8c292, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:32.680352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf787c7e-6ec1-4cf0-a624-af99ccd77eb0 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:33.187007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 887/1788] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:32 2022] PUT /identity/v3/projects/3626d81cf984436aa570cdb0171723ad/users/b7cdfb59a7904cb793bf94af35d8c292/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:33.202474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4601d-51ca-4aac-b5f3-9b6462badc61 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:33.202899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4601d-51ca-4aac-b5f3-9b6462badc61 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:33.203071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4601d-51ca-4aac-b5f3-9b6462badc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:33.280523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b2b4601d-51ca-4aac-b5f3-9b6462badc61 None None] MFA Rules not processed for user `b7cdfb59a7904cb793bf94af35d8c292`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:33.284600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b2b4601d-51ca-4aac-b5f3-9b6462badc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:33.509582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 902/1789] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:33.510832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf787c7e-6ec1-4cf0-a624-af99ccd77eb0 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:33.623272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf787c7e-6ec1-4cf0-a624-af99ccd77eb0 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:33.629954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228aeaf-5e22-479d-a685-8c3a81f42f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:33.630806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228aeaf-5e22-479d-a685-8c3a81f42f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:33.631088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228aeaf-5e22-479d-a685-8c3a81f42f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:33.631542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228aeaf-5e22-479d-a685-8c3a81f42f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:33.632787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228aeaf-5e22-479d-a685-8c3a81f42f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:33.636853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228aeaf-5e22-479d-a685-8c3a81f42f25 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:33.831153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 888/1790] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:33 2022] POST /identity/v3/projects => generated 383 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:52:33.837462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4601d-51ca-4aac-b5f3-9b6462badc61 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:33.880801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4601d-51ca-4aac-b5f3-9b6462badc61 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:34.221496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfbe76-f051-476f-9c78-dd89a3736b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:34.222660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfbe76-f051-476f-9c78-dd89a3736b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:34.223109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfbe76-f051-476f-9c78-dd89a3736b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:34.223543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfbe76-f051-476f-9c78-dd89a3736b87 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:34.224589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfbe76-f051-476f-9c78-dd89a3736b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:34.228574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfbe76-f051-476f-9c78-dd89a3736b87 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:34.367809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 903/1791] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:33 2022] POST /identity/v3/users => generated 292 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:52:34.371756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228aeaf-5e22-479d-a685-8c3a81f42f25 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:34.385555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228aeaf-5e22-479d-a685-8c3a81f42f25 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:34.404016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698902df-d0b7-4f4b-9ec7-66bc4f3b1658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:34.405103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698902df-d0b7-4f4b-9ec7-66bc4f3b1658 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:34.405619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698902df-d0b7-4f4b-9ec7-66bc4f3b1658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:34.406131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698902df-d0b7-4f4b-9ec7-66bc4f3b1658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:34.410670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698902df-d0b7-4f4b-9ec7-66bc4f3b1658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:34.413782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698902df-d0b7-4f4b-9ec7-66bc4f3b1658 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:34.463854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 889/1792] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:34.466474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfbe76-f051-476f-9c78-dd89a3736b87 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:34.493863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfbe76-f051-476f-9c78-dd89a3736b87 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:34.502819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb068c3-f031-49ab-9867-3f758b19d2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:34.503548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb068c3-f031-49ab-9867-3f758b19d2db admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:34.503816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb068c3-f031-49ab-9867-3f758b19d2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:34.504103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb068c3-f031-49ab-9867-3f758b19d2db admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:34.504840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb068c3-f031-49ab-9867-3f758b19d2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:34.510678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb068c3-f031-49ab-9867-3f758b19d2db admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:34.544744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 904/1793] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:34.549431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698902df-d0b7-4f4b-9ec7-66bc4f3b1658 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:34.565094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698902df-d0b7-4f4b-9ec7-66bc4f3b1658 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:34.576623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ca9e5-c1e4-44bb-b4bd-87ed88786323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:34.577734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca9e5-c1e4-44bb-b4bd-87ed88786323 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:34.578105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca9e5-c1e4-44bb-b4bd-87ed88786323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:34.578436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca9e5-c1e4-44bb-b4bd-87ed88786323 admin admin] PATH_INFO: `/v3/projects/db50fbb0748448a1ad14067ccf3eb8d2/users/6d4405e7b3ca43f4b67be0e530010095/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:34.647427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ca9e5-c1e4-44bb-b4bd-87ed88786323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db50fbb0748448a1ad14067ccf3eb8d2, user_id=6d4405e7b3ca43f4b67be0e530010095, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:34.650341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ca9e5-c1e4-44bb-b4bd-87ed88786323 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:34.679645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:34.746721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 890/1794] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:34 2022] PUT /identity/v3/projects/db50fbb0748448a1ad14067ccf3eb8d2/users/6d4405e7b3ca43f4b67be0e530010095/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:34.759650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8ddf-2a07-42d9-b5dc-5e1331a5ea7b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:34.759932 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8ddf-2a07-42d9-b5dc-5e1331a5ea7b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:34.760371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8ddf-2a07-42d9-b5dc-5e1331a5ea7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:34.894080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d8bf8ddf-2a07-42d9-b5dc-5e1331a5ea7b None None] MFA Rules not processed for user `6d4405e7b3ca43f4b67be0e530010095`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:34.894080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d8bf8ddf-2a07-42d9-b5dc-5e1331a5ea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:34.952606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5d0cf-46bd-44e3-bfe1-42a7ab9faa6f None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:34.957349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5d0cf-46bd-44e3-bfe1-42a7ab9faa6f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:34.957349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5d0cf-46bd-44e3-bfe1-42a7ab9faa6f None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:34.957867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 891/1795] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:52:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:52:35.030773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccb131-f2e7-4891-bde2-4fb1d04bfabc None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:35.031267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccb131-f2e7-4891-bde2-4fb1d04bfabc None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:35.031931 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccb131-f2e7-4891-bde2-4fb1d04bfabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:35.098389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e3ccb131-f2e7-4891-bde2-4fb1d04bfabc None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:35.100816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e3ccb131-f2e7-4891-bde2-4fb1d04bfabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:35.134789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 905/1796] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:35.139040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf8ddf-2a07-42d9-b5dc-5e1331a5ea7b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:35.268626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf8ddf-2a07-42d9-b5dc-5e1331a5ea7b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:35.278466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26381c1-b185-426c-90e6-861b2f77593c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:35.279029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26381c1-b185-426c-90e6-861b2f77593c admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:35.279531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26381c1-b185-426c-90e6-861b2f77593c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:35.280321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26381c1-b185-426c-90e6-861b2f77593c admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:35.281337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26381c1-b185-426c-90e6-861b2f77593c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:35.283826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26381c1-b185-426c-90e6-861b2f77593c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:35.364834 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 892/1797] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:52:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:35.427003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 906/1798] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:35 2022] POST /identity/v3/projects => generated 383 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:35.432158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccb131-f2e7-4891-bde2-4fb1d04bfabc None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:35.462697 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccb131-f2e7-4891-bde2-4fb1d04bfabc None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:35.572966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:35.665501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc608eb-8efa-4615-8314-820ded149754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:35.666132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc608eb-8efa-4615-8314-820ded149754 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:35.666413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc608eb-8efa-4615-8314-820ded149754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:35.667176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc608eb-8efa-4615-8314-820ded149754 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:35.667963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc608eb-8efa-4615-8314-820ded149754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:35.670090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc608eb-8efa-4615-8314-820ded149754 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:35.880257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 893/1799] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:35 2022] POST /identity/v3/users => generated 298 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:35.883149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26381c1-b185-426c-90e6-861b2f77593c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:35.906763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26381c1-b185-426c-90e6-861b2f77593c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:35.936693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fdafb-997a-4bf9-97e9-b8cd061ca779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:35.937379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fdafb-997a-4bf9-97e9-b8cd061ca779 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:35.937668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fdafb-997a-4bf9-97e9-b8cd061ca779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:35.938026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fdafb-997a-4bf9-97e9-b8cd061ca779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:35.938734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fdafb-997a-4bf9-97e9-b8cd061ca779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:35.950148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fdafb-997a-4bf9-97e9-b8cd061ca779 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:35.983344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 907/1800] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:35.994643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc608eb-8efa-4615-8314-820ded149754 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:36.012355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc608eb-8efa-4615-8314-820ded149754 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:36.025488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e5f06-9cdb-4abe-a7ff-af1b03e898dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:36.026363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e5f06-9cdb-4abe-a7ff-af1b03e898dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:36.027440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e5f06-9cdb-4abe-a7ff-af1b03e898dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:36.030580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e5f06-9cdb-4abe-a7ff-af1b03e898dc admin admin] PATH_INFO: `/v3/projects/f1391d325a2b4e12a416a319287aea4d/users/cfd979b0965644a09902b65cdfa6a91a/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:36.074225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e5f06-9cdb-4abe-a7ff-af1b03e898dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1391d325a2b4e12a416a319287aea4d, user_id=cfd979b0965644a09902b65cdfa6a91a, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:36.082844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e5f06-9cdb-4abe-a7ff-af1b03e898dc admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:36.213005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 894/1801] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:35 2022] PUT /identity/v3/projects/f1391d325a2b4e12a416a319287aea4d/users/cfd979b0965644a09902b65cdfa6a91a/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:52:36.217356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fdafb-997a-4bf9-97e9-b8cd061ca779 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:36.231232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fdafb-997a-4bf9-97e9-b8cd061ca779 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:36.250890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e5f06-9cdb-4abe-a7ff-af1b03e898dc admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:36.252657 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e70e5f06-9cdb-4abe-a7ff-af1b03e898dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:36.483820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a355b-e6de-486d-9299-dd4e1592dda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:36.484759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a355b-e6de-486d-9299-dd4e1592dda0 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:36.485264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a355b-e6de-486d-9299-dd4e1592dda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:36.485687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a355b-e6de-486d-9299-dd4e1592dda0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:36.486540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a355b-e6de-486d-9299-dd4e1592dda0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:36.488361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a355b-e6de-486d-9299-dd4e1592dda0 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:36.509905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 908/1802] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:36.511846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a355b-e6de-486d-9299-dd4e1592dda0 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:36.535965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a355b-e6de-486d-9299-dd4e1592dda0 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:36.545788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e61b1c-caeb-4af3-bfcc-5c083bb312db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:36.546708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e61b1c-caeb-4af3-bfcc-5c083bb312db admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:36.547344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e61b1c-caeb-4af3-bfcc-5c083bb312db admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:36.547763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e61b1c-caeb-4af3-bfcc-5c083bb312db admin admin] PATH_INFO: `/v3/projects/f1391d325a2b4e12a416a319287aea4d/users/cfd979b0965644a09902b65cdfa6a91a/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:36.549946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e61b1c-caeb-4af3-bfcc-5c083bb312db admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1391d325a2b4e12a416a319287aea4d, user_id=cfd979b0965644a09902b65cdfa6a91a, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:36.553631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e61b1c-caeb-4af3-bfcc-5c083bb312db admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:36.567774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 909/1803] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:36 2022] PUT /identity/v3/projects/f1391d325a2b4e12a416a319287aea4d/users/cfd979b0965644a09902b65cdfa6a91a/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:36.581587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a16da-fe1a-4948-a9db-8f147df8b2b9 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:36.582014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a16da-fe1a-4948-a9db-8f147df8b2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:36.583864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a16da-fe1a-4948-a9db-8f147df8b2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:36.636138 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-260a16da-fe1a-4948-a9db-8f147df8b2b9 None None] MFA Rules not processed for user `cfd979b0965644a09902b65cdfa6a91a`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:36.638310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-260a16da-fe1a-4948-a9db-8f147df8b2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:36.640351 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e5f06-9cdb-4abe-a7ff-af1b03e898dc admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:36.827877 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b5966-a38c-40cb-aab9-03f68bffdc57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:36.834092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5966-a38c-40cb-aab9-03f68bffdc57 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:36.834092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5966-a38c-40cb-aab9-03f68bffdc57 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:36.837097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5966-a38c-40cb-aab9-03f68bffdc57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:36.859916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b5966-a38c-40cb-aab9-03f68bffdc57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:36.866149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b5966-a38c-40cb-aab9-03f68bffdc57 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:36.888414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 910/1804] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:36.898094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 895/1805] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:36.902555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a16da-fe1a-4948-a9db-8f147df8b2b9 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:36.904149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-260a16da-fe1a-4948-a9db-8f147df8b2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:37.056122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a16da-fe1a-4948-a9db-8f147df8b2b9 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:37.066721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd972c99-2226-40d9-b222-a1f355a58c68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:37.067599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd972c99-2226-40d9-b222-a1f355a58c68 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:37.067947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd972c99-2226-40d9-b222-a1f355a58c68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:37.068338 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd972c99-2226-40d9-b222-a1f355a58c68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:37.222900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd972c99-2226-40d9-b222-a1f355a58c68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:37.305516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd972c99-2226-40d9-b222-a1f355a58c68 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:37.305516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 911/1806] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:38.435497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5170fb-da8e-4f56-a39d-e935cbf9d3b0 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:38.437557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5170fb-da8e-4f56-a39d-e935cbf9d3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:38.438671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5170fb-da8e-4f56-a39d-e935cbf9d3b0 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:38.442372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 896/1807] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:52:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:52:38.456892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8fcc9-e096-40ed-968c-c11e55e98ed8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:38.458429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8fcc9-e096-40ed-968c-c11e55e98ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:38.458429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8fcc9-e096-40ed-968c-c11e55e98ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:38.514151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5ad8fcc9-e096-40ed-968c-c11e55e98ed8 None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:38.514151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5ad8fcc9-e096-40ed-968c-c11e55e98ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:38.542952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 912/1808] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:52:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:38.575280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5170fb-da8e-4f56-a39d-e935cbf9d3b0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:38.586488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cb5170fb-da8e-4f56-a39d-e935cbf9d3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:38.640900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:38.673028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5170fb-da8e-4f56-a39d-e935cbf9d3b0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:38.690222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff73c1-9c00-47c5-9957-881e0c70086a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:38.695838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff73c1-9c00-47c5-9957-881e0c70086a service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:38.696627 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff73c1-9c00-47c5-9957-881e0c70086a service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:38.697077 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff73c1-9c00-47c5-9957-881e0c70086a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:38.710792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff73c1-9c00-47c5-9957-881e0c70086a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:38.713557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff73c1-9c00-47c5-9957-881e0c70086a service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:38.740983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 897/1809] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:52:39.930150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:40.192448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8fcc9-e096-40ed-968c-c11e55e98ed8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:40.194385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5ad8fcc9-e096-40ed-968c-c11e55e98ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:40.325304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:40.389124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8fcc9-e096-40ed-968c-c11e55e98ed8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:40.394725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:40.395392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:40.395674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:40.396007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] PATH_INFO: `/v3/users/24e4a337135a483ebfa25813913e376c` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:40.398733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e4a337135a483ebfa25813913e376c)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:40.404689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:40.627007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e4a337135a483ebfa25813913e376c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:40.627702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e4a337135a483ebfa25813913e376c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:40.702563 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e4a337135a483ebfa25813913e376c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:40.714801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e4a337135a483ebfa25813913e376c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:40.765467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e4a337135a483ebfa25813913e376c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:40.793895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 913/1810] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:40 2022] DELETE /identity/v3/users/24e4a337135a483ebfa25813913e376c => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:40.798359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f4a1-6a05-47a5-83c8-4b5c8c2c83ee None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:40.798684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f4a1-6a05-47a5-83c8-4b5c8c2c83ee None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:40.798958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f4a1-6a05-47a5-83c8-4b5c8c2c83ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:40.839584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-8862f4a1-6a05-47a5-83c8-4b5c8c2c83ee None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:40.841784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8862f4a1-6a05-47a5-83c8-4b5c8c2c83ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:41.012898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 898/1811] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:41.051692 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:41.056482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:41.199120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862f4a1-6a05-47a5-83c8-4b5c8c2c83ee None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:41.200353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e362532-9d0f-4113-927b-e0dd1cdd61fb admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:41.203302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8862f4a1-6a05-47a5-83c8-4b5c8c2c83ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:41.205238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cf766-9b9d-47bf-9e8d-71d044d594b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:41.206568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cf766-9b9d-47bf-9e8d-71d044d594b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:41.206568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cf766-9b9d-47bf-9e8d-71d044d594b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:41.207003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cf766-9b9d-47bf-9e8d-71d044d594b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:41.221134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cf766-9b9d-47bf-9e8d-71d044d594b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:41.223011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cf766-9b9d-47bf-9e8d-71d044d594b6 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:41.235407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 914/1812] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:41.374120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862f4a1-6a05-47a5-83c8-4b5c8c2c83ee None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:41.386995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6fd6e-89cc-439c-b660-d803d7150ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:41.389378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6fd6e-89cc-439c-b660-d803d7150ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:41.395691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6fd6e-89cc-439c-b660-d803d7150ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:41.397286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6fd6e-89cc-439c-b660-d803d7150ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:41.400143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-23c6fd6e-89cc-439c-b660-d803d7150ec6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:41.577053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6fd6e-89cc-439c-b660-d803d7150ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:41.582375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6fd6e-89cc-439c-b660-d803d7150ec6 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:41.623948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 899/1813] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:52:42.671108 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cf766-9b9d-47bf-9e8d-71d044d594b6 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:42.674363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f11cf766-9b9d-47bf-9e8d-71d044d594b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:42.698682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cf766-9b9d-47bf-9e8d-71d044d594b6 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:42.707205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:42.707904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:42.708975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:42.709415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] PATH_INFO: `/v3/projects/2ec9afcce98943b8aa21dfefd305d4a8` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:42.712068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec9afcce98943b8aa21dfefd305d4a8)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:42.714256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:42.886899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec9afcce98943b8aa21dfefd305d4a8', 'request_id': 'req-4ee7c023-3c88-4484-b0d4-e2216ffd126c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:42.910616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec9afcce98943b8aa21dfefd305d4a8', 'request_id': 'req-4ee7c023-3c88-4484-b0d4-e2216ffd126c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:43.024277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec9afcce98943b8aa21dfefd305d4a8', 'request_id': 'req-4ee7c023-3c88-4484-b0d4-e2216ffd126c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:43.049442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] The token cache is being invalidate because project 2ec9afcce98943b8aa21dfefd305d4a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:43.050053 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:43.057547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:43.057547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 915/1814] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:42 2022] DELETE /identity/v3/projects/2ec9afcce98943b8aa21dfefd305d4a8 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:43.057869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6fd6e-89cc-439c-b660-d803d7150ec6 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:43.083605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6fd6e-89cc-439c-b660-d803d7150ec6 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:43.088458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:43.256724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:INSERT=6,keystone:DELETE=1,keystone:SELECT=93 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:43.311339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:43.312425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:43.312930 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:43.313989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] PATH_INFO: `/v3/users/2a63adf7d5a74478bdfcfe0b4ce53fb9` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:43.315784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a63adf7d5a74478bdfcfe0b4ce53fb9)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:43.320510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:43.724535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a63adf7d5a74478bdfcfe0b4ce53fb9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:43.850462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a63adf7d5a74478bdfcfe0b4ce53fb9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:43.878928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a63adf7d5a74478bdfcfe0b4ce53fb9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:43.889523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a63adf7d5a74478bdfcfe0b4ce53fb9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:43.908373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a63adf7d5a74478bdfcfe0b4ce53fb9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:43.929578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 900/1815] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:43 2022] DELETE /identity/v3/users/2a63adf7d5a74478bdfcfe0b4ce53fb9 => generated 0 bytes in 872 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:44.255017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:44.256557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:44.373025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7c023-3c88-4484-b0d4-e2216ffd126c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:44.442623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcddad-19d1-4c0e-a7e9-d542302cded4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:44.443547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcddad-19d1-4c0e-a7e9-d542302cded4 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:44.444054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcddad-19d1-4c0e-a7e9-d542302cded4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:44.444560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcddad-19d1-4c0e-a7e9-d542302cded4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:44.446702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-36dcddad-19d1-4c0e-a7e9-d542302cded4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:44.527824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcddad-19d1-4c0e-a7e9-d542302cded4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:44.530095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcddad-19d1-4c0e-a7e9-d542302cded4 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:44.546106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 916/1816] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:45.130665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:45.130665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:45.261862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2a5eb-0246-4585-947b-07bf62925ce9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:45.269127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:45.270362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:45.270774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:45.271276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] PATH_INFO: `/v3/projects/644fc16140324c7f89dc35f016cd5add` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:45.272695 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=644fc16140324c7f89dc35f016cd5add)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:45.274966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:45.458558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644fc16140324c7f89dc35f016cd5add', 'request_id': 'req-a79db8ef-924e-4601-8afb-dc05a377ddfa'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:45.563525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644fc16140324c7f89dc35f016cd5add', 'request_id': 'req-a79db8ef-924e-4601-8afb-dc05a377ddfa'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:45.570886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644fc16140324c7f89dc35f016cd5add', 'request_id': 'req-a79db8ef-924e-4601-8afb-dc05a377ddfa'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:45.581036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] The token cache is being invalidate because project 644fc16140324c7f89dc35f016cd5add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:45.581327 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:45.581970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a79db8ef-924e-4601-8afb-dc05a377ddfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:45.584309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 901/1817] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:45 2022] DELETE /identity/v3/projects/644fc16140324c7f89dc35f016cd5add => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:45.589149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f3cda-892f-44e5-809b-a47565511e7e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:45.589609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f3cda-892f-44e5-809b-a47565511e7e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:45.589698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f3cda-892f-44e5-809b-a47565511e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:45.609773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6b3f3cda-892f-44e5-809b-a47565511e7e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:45.611003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6b3f3cda-892f-44e5-809b-a47565511e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:45.670703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 917/1818] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:45.678415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3574eb-6eb9-4397-9fd0-0508b487202a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:45.678815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3574eb-6eb9-4397-9fd0-0508b487202a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:45.679099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3574eb-6eb9-4397-9fd0-0508b487202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:45.706212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-cb3574eb-6eb9-4397-9fd0-0508b487202a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:45.709147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cb3574eb-6eb9-4397-9fd0-0508b487202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:45.717438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 902/1819] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:45.721785 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f3cda-892f-44e5-809b-a47565511e7e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:45.731530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f3cda-892f-44e5-809b-a47565511e7e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:45.736209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56193e73-56c5-4905-b46b-324ce9f17c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:45.736819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56193e73-56c5-4905-b46b-324ce9f17c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:45.737107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56193e73-56c5-4905-b46b-324ce9f17c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:45.737382 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56193e73-56c5-4905-b46b-324ce9f17c8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:45.738029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56193e73-56c5-4905-b46b-324ce9f17c8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:45.738336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56193e73-56c5-4905-b46b-324ce9f17c8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:45.743588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56193e73-56c5-4905-b46b-324ce9f17c8a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:45.760810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 918/1820] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:52:45.762058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3574eb-6eb9-4397-9fd0-0508b487202a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:45.769369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:45.774355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3574eb-6eb9-4397-9fd0-0508b487202a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:45.779491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087c723-2582-4462-95d5-0d5215b8c753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:45.780184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087c723-2582-4462-95d5-0d5215b8c753 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:45.780534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087c723-2582-4462-95d5-0d5215b8c753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:45.782296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087c723-2582-4462-95d5-0d5215b8c753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:45.782296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087c723-2582-4462-95d5-0d5215b8c753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:45.784043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087c723-2582-4462-95d5-0d5215b8c753 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:45.889784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 903/1821] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:45 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:45.897659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56193e73-56c5-4905-b46b-324ce9f17c8a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:45.910811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56193e73-56c5-4905-b46b-324ce9f17c8a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:46.030054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e21966-0bc6-4664-aef0-734ee4dd5749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:46.030054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e21966-0bc6-4664-aef0-734ee4dd5749 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:46.030054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e21966-0bc6-4664-aef0-734ee4dd5749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:46.030054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e21966-0bc6-4664-aef0-734ee4dd5749 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:46.030451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e21966-0bc6-4664-aef0-734ee4dd5749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:46.030451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e21966-0bc6-4664-aef0-734ee4dd5749 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:46.121591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 919/1822] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:45 2022] POST /identity/v3/users => generated 284 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:46.125637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087c723-2582-4462-95d5-0d5215b8c753 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:46.136668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087c723-2582-4462-95d5-0d5215b8c753 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:46.141250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a980c-6cb1-4726-a3da-96b85fa38207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:46.144544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a980c-6cb1-4726-a3da-96b85fa38207 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:46.144544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a980c-6cb1-4726-a3da-96b85fa38207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:46.144544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a980c-6cb1-4726-a3da-96b85fa38207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:46.144544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a980c-6cb1-4726-a3da-96b85fa38207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:46.145135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a980c-6cb1-4726-a3da-96b85fa38207 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:46.155677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 904/1823] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:46.159909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e21966-0bc6-4664-aef0-734ee4dd5749 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:46.170697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e21966-0bc6-4664-aef0-734ee4dd5749 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:46.175799 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e03fba-85e1-4155-912c-4fa29074c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:46.176781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e03fba-85e1-4155-912c-4fa29074c8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:46.177078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e03fba-85e1-4155-912c-4fa29074c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:46.177366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e03fba-85e1-4155-912c-4fa29074c8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:46.178015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e03fba-85e1-4155-912c-4fa29074c8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:46.179674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e03fba-85e1-4155-912c-4fa29074c8b6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:46.190325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 920/1824] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:46.195170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a980c-6cb1-4726-a3da-96b85fa38207 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:46.206962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a980c-6cb1-4726-a3da-96b85fa38207 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:46.212323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c95d9-5ef9-45d5-ad60-70c42b4b15b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:46.212981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c95d9-5ef9-45d5-ad60-70c42b4b15b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:46.213267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c95d9-5ef9-45d5-ad60-70c42b4b15b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:46.213550 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c95d9-5ef9-45d5-ad60-70c42b4b15b9 admin admin] PATH_INFO: `/v3/projects/f45d038b0137418e866bf1a3d41ecf4a/users/c19a0a7a17134d04a69cb2ab1f857f48/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:46.228684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c95d9-5ef9-45d5-ad60-70c42b4b15b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f45d038b0137418e866bf1a3d41ecf4a, user_id=c19a0a7a17134d04a69cb2ab1f857f48, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:46.231421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c95d9-5ef9-45d5-ad60-70c42b4b15b9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:46.358741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 905/1825] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:46 2022] PUT /identity/v3/projects/f45d038b0137418e866bf1a3d41ecf4a/users/c19a0a7a17134d04a69cb2ab1f857f48/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:46.367831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477614f-8a42-4aa3-890e-a1a710f938c8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:46.368718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477614f-8a42-4aa3-890e-a1a710f938c8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:46.368718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477614f-8a42-4aa3-890e-a1a710f938c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:46.401926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-1477614f-8a42-4aa3-890e-a1a710f938c8 None None] MFA Rules not processed for user `c19a0a7a17134d04a69cb2ab1f857f48`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:46.403176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1477614f-8a42-4aa3-890e-a1a710f938c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:46.468040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 921/1826] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:46.472448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c95d9-5ef9-45d5-ad60-70c42b4b15b9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:46.546226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c95d9-5ef9-45d5-ad60-70c42b4b15b9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:46.551986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12e643-4c59-4941-8ffb-0dcae57c949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:46.552601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12e643-4c59-4941-8ffb-0dcae57c949d admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:46.552888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12e643-4c59-4941-8ffb-0dcae57c949d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:46.553166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12e643-4c59-4941-8ffb-0dcae57c949d admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:46.553805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12e643-4c59-4941-8ffb-0dcae57c949d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:46.555759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12e643-4c59-4941-8ffb-0dcae57c949d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:46.691503 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 906/1827] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:46 2022] POST /identity/v3/projects => generated 367 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:46.697387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477614f-8a42-4aa3-890e-a1a710f938c8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:46.708828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477614f-8a42-4aa3-890e-a1a710f938c8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:46.774094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aeedf-b8f4-4d9e-994c-f8188ba96779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:46.774830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aeedf-b8f4-4d9e-994c-f8188ba96779 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:46.775182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aeedf-b8f4-4d9e-994c-f8188ba96779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:46.775525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aeedf-b8f4-4d9e-994c-f8188ba96779 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:46.776241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aeedf-b8f4-4d9e-994c-f8188ba96779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:46.778251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aeedf-b8f4-4d9e-994c-f8188ba96779 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:46.901878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 922/1828] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:46 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:52:46.907734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12e643-4c59-4941-8ffb-0dcae57c949d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:46.918126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12e643-4c59-4941-8ffb-0dcae57c949d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:46.922904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b5f5e-3545-4f56-8396-b53b917edd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:46.923519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b5f5e-3545-4f56-8396-b53b917edd1c admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:46.923805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b5f5e-3545-4f56-8396-b53b917edd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:46.924083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b5f5e-3545-4f56-8396-b53b917edd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:46.924762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b5f5e-3545-4f56-8396-b53b917edd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:46.927001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b5f5e-3545-4f56-8396-b53b917edd1c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:46.938057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 907/1829] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:46.943090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aeedf-b8f4-4d9e-994c-f8188ba96779 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:46.953645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aeedf-b8f4-4d9e-994c-f8188ba96779 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:46.959805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763526c7-28b9-4d50-9127-a52a393f9344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:46.960492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763526c7-28b9-4d50-9127-a52a393f9344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:46.960847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763526c7-28b9-4d50-9127-a52a393f9344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:46.961223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763526c7-28b9-4d50-9127-a52a393f9344 admin admin] PATH_INFO: `/v3/projects/ad13ac49bc9b4776bc85774526b25124/users/d7d4f75313aa4f848ef5d2f0da4a195a/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:46.980829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763526c7-28b9-4d50-9127-a52a393f9344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad13ac49bc9b4776bc85774526b25124, user_id=d7d4f75313aa4f848ef5d2f0da4a195a, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:46.983859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763526c7-28b9-4d50-9127-a52a393f9344 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:47.125910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 923/1830] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:46 2022] PUT /identity/v3/projects/ad13ac49bc9b4776bc85774526b25124/users/d7d4f75313aa4f848ef5d2f0da4a195a/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:47.126016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b5f5e-3545-4f56-8396-b53b917edd1c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:47.134146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b5f5e-3545-4f56-8396-b53b917edd1c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:47.205982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518df5e7-6dc1-41dc-b5f7-8489e9fdcc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:47.205982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518df5e7-6dc1-41dc-b5f7-8489e9fdcc2c admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:47.205982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518df5e7-6dc1-41dc-b5f7-8489e9fdcc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:47.205982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518df5e7-6dc1-41dc-b5f7-8489e9fdcc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:47.206945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518df5e7-6dc1-41dc-b5f7-8489e9fdcc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:47.207675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518df5e7-6dc1-41dc-b5f7-8489e9fdcc2c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:47.218552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 908/1831] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:52:47.223974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763526c7-28b9-4d50-9127-a52a393f9344 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:47.238319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763526c7-28b9-4d50-9127-a52a393f9344 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:47.251711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c543026-f50c-47da-a86f-24ffe6fbcbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:47.251711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c543026-f50c-47da-a86f-24ffe6fbcbdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:47.251711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c543026-f50c-47da-a86f-24ffe6fbcbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:47.251711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c543026-f50c-47da-a86f-24ffe6fbcbdf admin admin] PATH_INFO: `/v3/projects/ad13ac49bc9b4776bc85774526b25124/users/d7d4f75313aa4f848ef5d2f0da4a195a/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:47.252457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c543026-f50c-47da-a86f-24ffe6fbcbdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad13ac49bc9b4776bc85774526b25124, user_id=d7d4f75313aa4f848ef5d2f0da4a195a, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:47.258737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c543026-f50c-47da-a86f-24ffe6fbcbdf admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:47.279925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 924/1832] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:47 2022] PUT /identity/v3/projects/ad13ac49bc9b4776bc85774526b25124/users/d7d4f75313aa4f848ef5d2f0da4a195a/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:47.293886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931d7cf-d01e-49cd-9327-67ac238e694c None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:47.294451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931d7cf-d01e-49cd-9327-67ac238e694c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:47.294805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931d7cf-d01e-49cd-9327-67ac238e694c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:47.327078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-3931d7cf-d01e-49cd-9327-67ac238e694c None None] MFA Rules not processed for user `d7d4f75313aa4f848ef5d2f0da4a195a`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:47.329721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3931d7cf-d01e-49cd-9327-67ac238e694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:47.395850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 909/1833] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:47.408432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c543026-f50c-47da-a86f-24ffe6fbcbdf admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:47.411085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1c543026-f50c-47da-a86f-24ffe6fbcbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:47.471082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c543026-f50c-47da-a86f-24ffe6fbcbdf admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:47.476309 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13ef22-ba50-4388-995e-9783eed4a8d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:47.477281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13ef22-ba50-4388-995e-9783eed4a8d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:47.477701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13ef22-ba50-4388-995e-9783eed4a8d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:47.478111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13ef22-ba50-4388-995e-9783eed4a8d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:47.489780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13ef22-ba50-4388-995e-9783eed4a8d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:47.491434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13ef22-ba50-4388-995e-9783eed4a8d4 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:47.501349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 925/1834] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:47.735512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:47.743050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931d7cf-d01e-49cd-9327-67ac238e694c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:47.761131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931d7cf-d01e-49cd-9327-67ac238e694c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:47.767629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039afc16-69d6-4191-ac59-408edf7e3c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:47.768719 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039afc16-69d6-4191-ac59-408edf7e3c2e service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:47.769395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039afc16-69d6-4191-ac59-408edf7e3c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:47.770233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039afc16-69d6-4191-ac59-408edf7e3c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:47.869647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039afc16-69d6-4191-ac59-408edf7e3c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:47.871302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039afc16-69d6-4191-ac59-408edf7e3c2e service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:47.882605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 910/1835] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:49.711043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13ef22-ba50-4388-995e-9783eed4a8d4 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:49.715258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6d13ef22-ba50-4388-995e-9783eed4a8d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:49.778095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13ef22-ba50-4388-995e-9783eed4a8d4 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:49.783073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:49.783722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:49.784007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:49.784369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] PATH_INFO: `/v3/users/b7cdfb59a7904cb793bf94af35d8c292` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:49.786442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7cdfb59a7904cb793bf94af35d8c292)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:49.788390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:49.999469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7cdfb59a7904cb793bf94af35d8c292'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:50.000196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7cdfb59a7904cb793bf94af35d8c292'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:50.291750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7cdfb59a7904cb793bf94af35d8c292'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:50.295690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:50.307616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7cdfb59a7904cb793bf94af35d8c292'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:50.348218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7cdfb59a7904cb793bf94af35d8c292'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:50.363610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 926/1836] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:49 2022] DELETE /identity/v3/users/b7cdfb59a7904cb793bf94af35d8c292 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:50.368531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebb7ed-eaaf-4978-84bd-f14f168b8bce None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:50.368818 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebb7ed-eaaf-4978-84bd-f14f168b8bce None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:50.369099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebb7ed-eaaf-4978-84bd-f14f168b8bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:50.414763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f2ebb7ed-eaaf-4978-84bd-f14f168b8bce None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:50.416014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f2ebb7ed-eaaf-4978-84bd-f14f168b8bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:50.599847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 911/1837] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:50.609580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:50.611145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:50.700711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcfb187-877c-439a-9667-71031843f1ad admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:50.704946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712731b0-d0d0-4918-ab7a-faeb008194f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:50.705736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712731b0-d0d0-4918-ab7a-faeb008194f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:50.708486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712731b0-d0d0-4918-ab7a-faeb008194f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:50.708486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712731b0-d0d0-4918-ab7a-faeb008194f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:50.718877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712731b0-d0d0-4918-ab7a-faeb008194f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:50.720504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712731b0-d0d0-4918-ab7a-faeb008194f7 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:50.733658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 927/1838] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:50.768377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:50.806764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903e414-3872-419a-9c50-1cf9a6649cb3 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:50.806764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903e414-3872-419a-9c50-1cf9a6649cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:50.806764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903e414-3872-419a-9c50-1cf9a6649cb3 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:50.806764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 912/1839] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:52:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:52:50.819453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaf574-2c11-41c8-b2b3-245782684b9c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:50.822074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaf574-2c11-41c8-b2b3-245782684b9c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:50.822411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaf574-2c11-41c8-b2b3-245782684b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:50.876496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-2afaf574-2c11-41c8-b2b3-245782684b9c None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:50.881430 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2afaf574-2c11-41c8-b2b3-245782684b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:51.047372 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 928/1840] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:52:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:51.063348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903e414-3872-419a-9c50-1cf9a6649cb3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:51.066639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c903e414-3872-419a-9c50-1cf9a6649cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:51.088266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903e414-3872-419a-9c50-1cf9a6649cb3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:51.094352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebf18d-393b-4dc0-a7c9-8abb3bd0936b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:51.095014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebf18d-393b-4dc0-a7c9-8abb3bd0936b service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:51.095328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebf18d-393b-4dc0-a7c9-8abb3bd0936b service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:51.095625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebf18d-393b-4dc0-a7c9-8abb3bd0936b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:51.106100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebf18d-393b-4dc0-a7c9-8abb3bd0936b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:51.107662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebf18d-393b-4dc0-a7c9-8abb3bd0936b service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:51.124197 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 913/1841] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:51.575038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaf574-2c11-41c8-b2b3-245782684b9c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:51.577228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2afaf574-2c11-41c8-b2b3-245782684b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:51.599859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaf574-2c11-41c8-b2b3-245782684b9c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:51.619836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:51.620758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:51.621154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:51.621593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] PATH_INFO: `/v3/projects/3626d81cf984436aa570cdb0171723ad` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:51.623646 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3626d81cf984436aa570cdb0171723ad)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:51.639543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:51.683464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:51.831930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3626d81cf984436aa570cdb0171723ad', 'request_id': 'req-4ff127c7-219b-4075-a773-9671c6ce9a86'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:51.840055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3626d81cf984436aa570cdb0171723ad', 'request_id': 'req-4ff127c7-219b-4075-a773-9671c6ce9a86'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:51.934810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3626d81cf984436aa570cdb0171723ad', 'request_id': 'req-4ff127c7-219b-4075-a773-9671c6ce9a86'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:51.953876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] The token cache is being invalidate because project 3626d81cf984436aa570cdb0171723ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:51.955128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:51.956772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:51.959334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 929/1842] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:51 2022] DELETE /identity/v3/projects/3626d81cf984436aa570cdb0171723ad => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:51.963416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebf18d-393b-4dc0-a7c9-8abb3bd0936b service nova] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:51.974560 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebf18d-393b-4dc0-a7c9-8abb3bd0936b service nova] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:52.050667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:52.051332 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:52.051617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:52.057524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] PATH_INFO: `/v3/users/6d4405e7b3ca43f4b67be0e530010095` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:52.057524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d4405e7b3ca43f4b67be0e530010095)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:52.057524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:52.340561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d4405e7b3ca43f4b67be0e530010095'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:52.396873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d4405e7b3ca43f4b67be0e530010095'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:52.413870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d4405e7b3ca43f4b67be0e530010095'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:52.421597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d4405e7b3ca43f4b67be0e530010095'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:52.422962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d4405e7b3ca43f4b67be0e530010095'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:52.434724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 914/1843] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:51 2022] DELETE /identity/v3/users/6d4405e7b3ca43f4b67be0e530010095 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:52.837165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:52.901285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:52.902788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:52.916583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff127c7-219b-4075-a773-9671c6ce9a86 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:52.992631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:52.993332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:52.993618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:52.993893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] PATH_INFO: `/v3/projects/db50fbb0748448a1ad14067ccf3eb8d2` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:52.997034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db50fbb0748448a1ad14067ccf3eb8d2)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:53.000179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:53.171380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db50fbb0748448a1ad14067ccf3eb8d2', 'request_id': 'req-2dc22a06-4c9e-4564-b860-da0ce41cae24'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.178467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db50fbb0748448a1ad14067ccf3eb8d2', 'request_id': 'req-2dc22a06-4c9e-4564-b860-da0ce41cae24'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.241672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db50fbb0748448a1ad14067ccf3eb8d2', 'request_id': 'req-2dc22a06-4c9e-4564-b860-da0ce41cae24'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.255530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] The token cache is being invalidate because project db50fbb0748448a1ad14067ccf3eb8d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:53.255831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.256592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.258392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 930/1844] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:52 2022] DELETE /identity/v3/projects/db50fbb0748448a1ad14067ccf3eb8d2 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:52:53.262445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:53.263906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:53.278721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc68ab4-8011-464f-9713-f1c817984016 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:53.345555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:53.346214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:53.347143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:53.347623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] PATH_INFO: `/v3/users/cfd979b0965644a09902b65cdfa6a91a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:53.349099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd979b0965644a09902b65cdfa6a91a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:53.351092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:53.575017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd979b0965644a09902b65cdfa6a91a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.647794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd979b0965644a09902b65cdfa6a91a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.661144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd979b0965644a09902b65cdfa6a91a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.667990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd979b0965644a09902b65cdfa6a91a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.668689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd979b0965644a09902b65cdfa6a91a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:53.674797 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 915/1845] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:53 2022] DELETE /identity/v3/users/cfd979b0965644a09902b65cdfa6a91a => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:52:53.756344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:53.757818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:53.762351 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:53.772723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc22a06-4c9e-4564-b860-da0ce41cae24 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:53.832748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:53.833534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:53.833825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:53.834102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] PATH_INFO: `/v3/users/3f81d93f3c4e4cf7a9d78c1c790567bb` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:53.835351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f81d93f3c4e4cf7a9d78c1c790567bb)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:53.837290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:53.945065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:54.083880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f81d93f3c4e4cf7a9d78c1c790567bb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.084623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f81d93f3c4e4cf7a9d78c1c790567bb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.110824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:54.113044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:54.127552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba59c5-23f9-4e4c-9488-6af169163a6f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:54.141127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:54.141127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:54.141127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:54.141127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] PATH_INFO: `/v3/projects/f1391d325a2b4e12a416a319287aea4d` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:54.141755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1391d325a2b4e12a416a319287aea4d)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:54.148830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:54.189592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f81d93f3c4e4cf7a9d78c1c790567bb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.199491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f81d93f3c4e4cf7a9d78c1c790567bb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.222727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-231b383e-7461-45ba-8d7f-d325d46c25c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f81d93f3c4e4cf7a9d78c1c790567bb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.222727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 931/1846] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:53 2022] DELETE /identity/v3/users/3f81d93f3c4e4cf7a9d78c1c790567bb => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:54.226192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05909916-e706-49de-ab99-9bb2d8065c04 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:54.226491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05909916-e706-49de-ab99-9bb2d8065c04 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:54.226842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05909916-e706-49de-ab99-9bb2d8065c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:54.247251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-05909916-e706-49de-ab99-9bb2d8065c04 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:54.248501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-05909916-e706-49de-ab99-9bb2d8065c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:54.296050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1391d325a2b4e12a416a319287aea4d', 'request_id': 'req-db761865-d346-4f91-aecd-0cbf47c7a52b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.325439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 932/1847] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:52:54.343530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05909916-e706-49de-ab99-9bb2d8065c04 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:54.345038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-05909916-e706-49de-ab99-9bb2d8065c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:54.390284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1391d325a2b4e12a416a319287aea4d', 'request_id': 'req-db761865-d346-4f91-aecd-0cbf47c7a52b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.398223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1391d325a2b4e12a416a319287aea4d', 'request_id': 'req-db761865-d346-4f91-aecd-0cbf47c7a52b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.412869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05909916-e706-49de-ab99-9bb2d8065c04 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:54.441793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] The token cache is being invalidate because project f1391d325a2b4e12a416a319287aea4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:54.442321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.443180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db761865-d346-4f91-aecd-0cbf47c7a52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:54.444935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 916/1848] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:54 2022] DELETE /identity/v3/projects/f1391d325a2b4e12a416a319287aea4d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:52:54.452418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff3806-dd64-44d5-bcee-4d6d93e32ae2 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:54.453177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff3806-dd64-44d5-bcee-4d6d93e32ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:54.453529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff3806-dd64-44d5-bcee-4d6d93e32ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:54.475922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-ccff3806-dd64-44d5-bcee-4d6d93e32ae2 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:54.477766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ccff3806-dd64-44d5-bcee-4d6d93e32ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:54.492891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf255c5-1067-4da1-abc5-1ed0f2da7b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:54.497855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf255c5-1067-4da1-abc5-1ed0f2da7b11 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:54.505230 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf255c5-1067-4da1-abc5-1ed0f2da7b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:54.506768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf255c5-1067-4da1-abc5-1ed0f2da7b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:54.517998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-acf255c5-1067-4da1-abc5-1ed0f2da7b11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:54.598160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 917/1849] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:54.600583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4ac1c-a7c6-43ad-9ca8-d334deec5dbd None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:54.601112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4ac1c-a7c6-43ad-9ca8-d334deec5dbd None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:54.601751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4ac1c-a7c6-43ad-9ca8-d334deec5dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:54.628925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b8b4ac1c-a7c6-43ad-9ca8-d334deec5dbd None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:54.630784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b8b4ac1c-a7c6-43ad-9ca8-d334deec5dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:54.644153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 918/1850] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:54.650612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4ac1c-a7c6-43ad-9ca8-d334deec5dbd None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:54.664604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf255c5-1067-4da1-abc5-1ed0f2da7b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:54.666240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf255c5-1067-4da1-abc5-1ed0f2da7b11 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:54.679182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4ac1c-a7c6-43ad-9ca8-d334deec5dbd None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:54.680250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 933/1851] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:52:54.685075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a349f-efdf-471e-9a46-2cf39b06ad27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:54.685832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a349f-efdf-471e-9a46-2cf39b06ad27 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:54.686224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a349f-efdf-471e-9a46-2cf39b06ad27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:54.686660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a349f-efdf-471e-9a46-2cf39b06ad27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:54.687432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a349f-efdf-471e-9a46-2cf39b06ad27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:54.687841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a349f-efdf-471e-9a46-2cf39b06ad27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:54.690871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a349f-efdf-471e-9a46-2cf39b06ad27 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:54.706371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 919/1852] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:52:54.711086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf255c5-1067-4da1-abc5-1ed0f2da7b11 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:54.724123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf255c5-1067-4da1-abc5-1ed0f2da7b11 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:54.730701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7056bb1-8f19-44d8-8320-2164add0dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:54.731532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7056bb1-8f19-44d8-8320-2164add0dc42 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:54.731925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7056bb1-8f19-44d8-8320-2164add0dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:54.732311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7056bb1-8f19-44d8-8320-2164add0dc42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:54.733115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7056bb1-8f19-44d8-8320-2164add0dc42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:54.736152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7056bb1-8f19-44d8-8320-2164add0dc42 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:54.925905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 934/1853] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:54 2022] POST /identity/v3/projects => generated 369 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:52:54.927532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a349f-efdf-471e-9a46-2cf39b06ad27 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:54.937810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a349f-efdf-471e-9a46-2cf39b06ad27 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:55.006808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3301a9-a11c-4d92-a941-645c526bc590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:55.007657 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3301a9-a11c-4d92-a941-645c526bc590 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:55.008014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3301a9-a11c-4d92-a941-645c526bc590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:55.008303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3301a9-a11c-4d92-a941-645c526bc590 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:55.008940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3301a9-a11c-4d92-a941-645c526bc590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:55.010963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3301a9-a11c-4d92-a941-645c526bc590 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:55.287565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 920/1854] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:54 2022] POST /identity/v3/users => generated 285 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:52:55.288367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7056bb1-8f19-44d8-8320-2164add0dc42 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:55.303831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7056bb1-8f19-44d8-8320-2164add0dc42 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:55.314110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d7c10-58f8-497e-b504-1e04ca1fc5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:55.316498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d7c10-58f8-497e-b504-1e04ca1fc5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:55.317036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d7c10-58f8-497e-b504-1e04ca1fc5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:55.317379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d7c10-58f8-497e-b504-1e04ca1fc5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:55.318132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d7c10-58f8-497e-b504-1e04ca1fc5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:55.320814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d7c10-58f8-497e-b504-1e04ca1fc5f7 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:55.344284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 935/1855] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:52:55.350409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3301a9-a11c-4d92-a941-645c526bc590 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:55.364870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3301a9-a11c-4d92-a941-645c526bc590 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:55.371232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed171e-fd07-4ced-9076-a7820a8ae8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:55.372087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed171e-fd07-4ced-9076-a7820a8ae8fa admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:55.372373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed171e-fd07-4ced-9076-a7820a8ae8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:55.372655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed171e-fd07-4ced-9076-a7820a8ae8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:55.373359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed171e-fd07-4ced-9076-a7820a8ae8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:55.375262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed171e-fd07-4ced-9076-a7820a8ae8fa admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:55.391656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d7c10-58f8-497e-b504-1e04ca1fc5f7 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:55.394582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-318d7c10-58f8-497e-b504-1e04ca1fc5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:55.418576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 921/1856] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:52:55.426319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d7c10-58f8-497e-b504-1e04ca1fc5f7 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:55.430128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed171e-fd07-4ced-9076-a7820a8ae8fa admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:55.435674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-daed171e-fd07-4ced-9076-a7820a8ae8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:55.443181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:55.444124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:55.444509 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:55.444968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] PATH_INFO: `/v3/projects/00e324df62b84929b831e49543331b8b` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:55.447294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e324df62b84929b831e49543331b8b)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:55.449480 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:55.481002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed171e-fd07-4ced-9076-a7820a8ae8fa admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:55.495456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:55.496110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:55.496391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:55.496664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] PATH_INFO: `/v3/users/c19a0a7a17134d04a69cb2ab1f857f48` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:55.497831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c19a0a7a17134d04a69cb2ab1f857f48)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:55.499756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:55.585723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e324df62b84929b831e49543331b8b', 'request_id': 'req-8937bd9d-a1bd-40a0-8702-2e8e16f89759'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:55.593212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e324df62b84929b831e49543331b8b', 'request_id': 'req-8937bd9d-a1bd-40a0-8702-2e8e16f89759'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:55.715854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e324df62b84929b831e49543331b8b', 'request_id': 'req-8937bd9d-a1bd-40a0-8702-2e8e16f89759'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:55.716165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:55.727365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] The token cache is being invalidate because project 00e324df62b84929b831e49543331b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:55.727654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:55.728342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:55.729846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 936/1857] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:55 2022] DELETE /identity/v3/projects/00e324df62b84929b831e49543331b8b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:52:55.730749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:55.732128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:55.797033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937bd9d-a1bd-40a0-8702-2e8e16f89759 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:55.804927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864670a-bb2e-4aa7-b2a5-1b5d5254a5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:55.805622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864670a-bb2e-4aa7-b2a5-1b5d5254a5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:55.806003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864670a-bb2e-4aa7-b2a5-1b5d5254a5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:55.806174 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864670a-bb2e-4aa7-b2a5-1b5d5254a5fe admin admin] PATH_INFO: `/v3/projects/dc09c90dfe2a43d7a32329ad903bbf26/users/3ff6642706b74460badcad0eed3c7de6/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:55.833186 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864670a-bb2e-4aa7-b2a5-1b5d5254a5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc09c90dfe2a43d7a32329ad903bbf26, user_id=3ff6642706b74460badcad0eed3c7de6, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:55.836072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864670a-bb2e-4aa7-b2a5-1b5d5254a5fe admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:55.848753 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c19a0a7a17134d04a69cb2ab1f857f48'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.133430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c19a0a7a17134d04a69cb2ab1f857f48'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.142139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 937/1858] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:55 2022] PUT /identity/v3/projects/dc09c90dfe2a43d7a32329ad903bbf26/users/3ff6642706b74460badcad0eed3c7de6/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:56.145325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864670a-bb2e-4aa7-b2a5-1b5d5254a5fe admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:56.146102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8864670a-bb2e-4aa7-b2a5-1b5d5254a5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:56.150432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c19a0a7a17134d04a69cb2ab1f857f48'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.158253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c19a0a7a17134d04a69cb2ab1f857f48'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.159198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e579493-7b33-461e-b52c-8166872f91b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c19a0a7a17134d04a69cb2ab1f857f48'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.169136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 922/1859] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:55 2022] DELETE /identity/v3/users/c19a0a7a17134d04a69cb2ab1f857f48 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:56.169136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d350ae8-a7ea-4c50-8b68-037eb1f844da None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:56.169136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d350ae8-a7ea-4c50-8b68-037eb1f844da None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:56.169136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d350ae8-a7ea-4c50-8b68-037eb1f844da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:56.199652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-1d350ae8-a7ea-4c50-8b68-037eb1f844da None None] MFA Rules not processed for user `3ff6642706b74460badcad0eed3c7de6`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:56.201025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1d350ae8-a7ea-4c50-8b68-037eb1f844da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:56.229656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864670a-bb2e-4aa7-b2a5-1b5d5254a5fe admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:56.270983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 923/1860] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:56.272358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088072aa-1a94-442a-b3e0-4f9d54aa41be None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:56.272654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088072aa-1a94-442a-b3e0-4f9d54aa41be None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:56.272937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088072aa-1a94-442a-b3e0-4f9d54aa41be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:56.294645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-088072aa-1a94-442a-b3e0-4f9d54aa41be None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:56.299872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-088072aa-1a94-442a-b3e0-4f9d54aa41be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:56.302425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:56.303129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:56.303400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:56.303681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] PATH_INFO: `/v3/users/08b46aa98c5a4f37824026c4fa32b062` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:56.304851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08b46aa98c5a4f37824026c4fa32b062)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:56.306847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:56.357073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 924/1861] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:56.357853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088072aa-1a94-442a-b3e0-4f9d54aa41be None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:56.359143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-088072aa-1a94-442a-b3e0-4f9d54aa41be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:56.444583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088072aa-1a94-442a-b3e0-4f9d54aa41be None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:56.448563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da473e52-f47b-4e55-9a07-3a25c26732d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ff3f53ddbb04eb8916b9ccfdb03dd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:56.449173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da473e52-f47b-4e55-9a07-3a25c26732d3 service glance] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:56.449458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da473e52-f47b-4e55-9a07-3a25c26732d3 service glance] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:56.449738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da473e52-f47b-4e55-9a07-3a25c26732d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:56.458935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da473e52-f47b-4e55-9a07-3a25c26732d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:56.460486 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da473e52-f47b-4e55-9a07-3a25c26732d3 service glance] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:56.469552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 925/1862] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:56.470458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da473e52-f47b-4e55-9a07-3a25c26732d3 service glance] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:56.471815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-da473e52-f47b-4e55-9a07-3a25c26732d3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:56.540305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da473e52-f47b-4e55-9a07-3a25c26732d3 service glance] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:56.544986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce269-6b13-428a-a3ac-beb53bafb677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:56.546832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce269-6b13-428a-a3ac-beb53bafb677 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:56.547002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce269-6b13-428a-a3ac-beb53bafb677 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:56.547282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce269-6b13-428a-a3ac-beb53bafb677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:56.553102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08b46aa98c5a4f37824026c4fa32b062'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.554168 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08b46aa98c5a4f37824026c4fa32b062'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.565111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ce269-6b13-428a-a3ac-beb53bafb677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:56.567450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ce269-6b13-428a-a3ac-beb53bafb677 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:56.583425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 926/1863] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:56.690193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08b46aa98c5a4f37824026c4fa32b062'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.700506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08b46aa98c5a4f37824026c4fa32b062'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.718626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08b46aa98c5a4f37824026c4fa32b062'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:56.740310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 938/1864] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:56 2022] DELETE /identity/v3/users/08b46aa98c5a4f37824026c4fa32b062 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:57.190527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:57.415542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce269-6b13-428a-a3ac-beb53bafb677 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:57.417046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2d1ce269-6b13-428a-a3ac-beb53bafb677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:57.433010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce269-6b13-428a-a3ac-beb53bafb677 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:57.490694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:57.491340 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:57.491628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:57.491901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] PATH_INFO: `/v3/projects/f45d038b0137418e866bf1a3d41ecf4a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:57.493045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f45d038b0137418e866bf1a3d41ecf4a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:57.495052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:57.543432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:57.545498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:57.567519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089db9a-9f53-4f2e-984f-b9447852e698 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:57.572864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:57.573547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:57.573832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:57.574206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] PATH_INFO: `/v3/projects/24c63591df1a473e9a877f2f8a9d48a9` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:57.575550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c63591df1a473e9a877f2f8a9d48a9)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:57.577659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:57.686526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f45d038b0137418e866bf1a3d41ecf4a', 'request_id': 'req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:57.713879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c63591df1a473e9a877f2f8a9d48a9', 'request_id': 'req-25cd1905-afe8-44d1-b379-6875bc27ec57'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:57.719678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c63591df1a473e9a877f2f8a9d48a9', 'request_id': 'req-25cd1905-afe8-44d1-b379-6875bc27ec57'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:57.820951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f45d038b0137418e866bf1a3d41ecf4a', 'request_id': 'req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:57.831475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f45d038b0137418e866bf1a3d41ecf4a', 'request_id': 'req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:57.849613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] The token cache is being invalidate because project f45d038b0137418e866bf1a3d41ecf4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:57.850017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:57.850868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:57.855176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 927/1865] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:57 2022] DELETE /identity/v3/projects/f45d038b0137418e866bf1a3d41ecf4a => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:57.856877 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:57.863397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:57.954433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a91c0d-94ad-48ab-bd4e-98d02759f6f2 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:57.978954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c63591df1a473e9a877f2f8a9d48a9', 'request_id': 'req-25cd1905-afe8-44d1-b379-6875bc27ec57'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:57.994245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] The token cache is being invalidate because project 24c63591df1a473e9a877f2f8a9d48a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:52:58.010198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:58.011607 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-25cd1905-afe8-44d1-b379-6875bc27ec57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:58.013674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 939/1866] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:52:57 2022] DELETE /identity/v3/projects/24c63591df1a473e9a877f2f8a9d48a9 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:58.024044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:58.025548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becca83-7785-466d-89a7-1c1f9f28e06b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:58.025847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becca83-7785-466d-89a7-1c1f9f28e06b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:58.026122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becca83-7785-466d-89a7-1c1f9f28e06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:58.057049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5becca83-7785-466d-89a7-1c1f9f28e06b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:58.058454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5becca83-7785-466d-89a7-1c1f9f28e06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:58.088496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:58.089377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:58.090071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:58.090537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] PATH_INFO: `/v3/users/d7d4f75313aa4f848ef5d2f0da4a195a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:58.091858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d4f75313aa4f848ef5d2f0da4a195a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:58.094633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:58.173287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 940/1867] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:58.175804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2e852-322a-42c8-8cca-6fd09f570e70 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:58.176263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2e852-322a-42c8-8cca-6fd09f570e70 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:58.176658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2e852-322a-42c8-8cca-6fd09f570e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:58.198835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-afd2e852-322a-42c8-8cca-6fd09f570e70 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:58.200406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-afd2e852-322a-42c8-8cca-6fd09f570e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:58.209469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 941/1868] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:58.215658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2e852-322a-42c8-8cca-6fd09f570e70 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:58.226424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2e852-322a-42c8-8cca-6fd09f570e70 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:58.234771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a775b5f-cddf-42fe-97cc-9dadf5560cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:58.235795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775b5f-cddf-42fe-97cc-9dadf5560cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:58.237811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775b5f-cddf-42fe-97cc-9dadf5560cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:58.238488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775b5f-cddf-42fe-97cc-9dadf5560cf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:58.240078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a775b5f-cddf-42fe-97cc-9dadf5560cf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:52:58.240829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a775b5f-cddf-42fe-97cc-9dadf5560cf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:58.243410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a775b5f-cddf-42fe-97cc-9dadf5560cf4 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:58.257207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 942/1869] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:52:58.261788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a775b5f-cddf-42fe-97cc-9dadf5560cf4 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:58.278243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a775b5f-cddf-42fe-97cc-9dadf5560cf4 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:58.284463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c1ae9-280c-4759-b7cb-5f516cdd6e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:58.285097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c1ae9-280c-4759-b7cb-5f516cdd6e98 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:58.285385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c1ae9-280c-4759-b7cb-5f516cdd6e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:58.285670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c1ae9-280c-4759-b7cb-5f516cdd6e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:58.286343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c1ae9-280c-4759-b7cb-5f516cdd6e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:58.288572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c1ae9-280c-4759-b7cb-5f516cdd6e98 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:58.382195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d4f75313aa4f848ef5d2f0da4a195a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:58.411069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 943/1870] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:52:58 2022] POST /identity/v3/projects => generated 389 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:52:58.415584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c1ae9-280c-4759-b7cb-5f516cdd6e98 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:58.446714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d4f75313aa4f848ef5d2f0da4a195a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:58.465509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d4f75313aa4f848ef5d2f0da4a195a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:58.476626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d4f75313aa4f848ef5d2f0da4a195a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:58.477912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d4f75313aa4f848ef5d2f0da4a195a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:52:58.491117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 928/1871] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:52:57 2022] DELETE /identity/v3/users/d7d4f75313aa4f848ef5d2f0da4a195a => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:52:58.512262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c1ae9-280c-4759-b7cb-5f516cdd6e98 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:58.604576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bc6f3-26ae-421d-b357-4f8ccce29d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:58.605215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bc6f3-26ae-421d-b357-4f8ccce29d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:58.605592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bc6f3-26ae-421d-b357-4f8ccce29d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:58.605877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bc6f3-26ae-421d-b357-4f8ccce29d89 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:58.606579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bc6f3-26ae-421d-b357-4f8ccce29d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:58.609243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bc6f3-26ae-421d-b357-4f8ccce29d89 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:59.190000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 944/1872] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:52:58 2022] POST /identity/v3/users => generated 295 bytes in 766 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:52:59.190221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:59.190221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:59.203706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199227b0-bb05-4a79-a792-19a17921de85 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:59.208892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af24ba-5e31-42f2-ba20-97dee7d5cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:59.209536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af24ba-5e31-42f2-ba20-97dee7d5cc85 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:59.209852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af24ba-5e31-42f2-ba20-97dee7d5cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:59.210139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af24ba-5e31-42f2-ba20-97dee7d5cc85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:59.211072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af24ba-5e31-42f2-ba20-97dee7d5cc85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:59.213006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af24ba-5e31-42f2-ba20-97dee7d5cc85 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:59.225649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 929/1873] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:59.230883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bc6f3-26ae-421d-b357-4f8ccce29d89 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:59.242311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bc6f3-26ae-421d-b357-4f8ccce29d89 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:59.249197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8328a-5d96-4d29-944c-81f82474f850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:59.249894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8328a-5d96-4d29-944c-81f82474f850 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:59.250128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8328a-5d96-4d29-944c-81f82474f850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:59.250375 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8328a-5d96-4d29-944c-81f82474f850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:59.251094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8328a-5d96-4d29-944c-81f82474f850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:59.252917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8328a-5d96-4d29-944c-81f82474f850 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:59.265285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 945/1874] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:52:59.285780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af24ba-5e31-42f2-ba20-97dee7d5cc85 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:59.297104 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af24ba-5e31-42f2-ba20-97dee7d5cc85 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:59.303647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2697f3-1d7e-4111-8ee9-372913237534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:59.306928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2697f3-1d7e-4111-8ee9-372913237534 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:59.307477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2697f3-1d7e-4111-8ee9-372913237534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:59.307854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2697f3-1d7e-4111-8ee9-372913237534 admin admin] PATH_INFO: `/v3/projects/5aad49c87c4449309f0947e70b486565/users/67beb33068e54bd8a98da10b42fcdae3/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:59.331924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2697f3-1d7e-4111-8ee9-372913237534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aad49c87c4449309f0947e70b486565, user_id=67beb33068e54bd8a98da10b42fcdae3, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:59.336078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2697f3-1d7e-4111-8ee9-372913237534 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:00.081890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 930/1875] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:52:59 2022] PUT /identity/v3/projects/5aad49c87c4449309f0947e70b486565/users/67beb33068e54bd8a98da10b42fcdae3/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 797 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:00.093341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31bcf-5bae-44d8-8d46-7f929c955a1a None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.093637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31bcf-5bae-44d8-8d46-7f929c955a1a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.093914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31bcf-5bae-44d8-8d46-7f929c955a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.136522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-46c31bcf-5bae-44d8-8d46-7f929c955a1a None None] MFA Rules not processed for user `67beb33068e54bd8a98da10b42fcdae3`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:00.138192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-46c31bcf-5bae-44d8-8d46-7f929c955a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:00.155886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2697f3-1d7e-4111-8ee9-372913237534 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:00.157677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-da2697f3-1d7e-4111-8ee9-372913237534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:00.215056 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 946/1876] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:00.219728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c31bcf-5bae-44d8-8d46-7f929c955a1a None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:00.254315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2697f3-1d7e-4111-8ee9-372913237534 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:00.259617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:00.260438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.261119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.261636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] PATH_INFO: `/v3/projects/ad13ac49bc9b4776bc85774526b25124` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.262946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad13ac49bc9b4776bc85774526b25124)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:00.265089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:00.305185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c31bcf-5bae-44d8-8d46-7f929c955a1a None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:00.309908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a621-c29e-4503-b737-4bde6e19acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:00.310612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a621-c29e-4503-b737-4bde6e19acff admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.310904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a621-c29e-4503-b737-4bde6e19acff admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.311182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a621-c29e-4503-b737-4bde6e19acff admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.311814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a621-c29e-4503-b737-4bde6e19acff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:00.314576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a621-c29e-4503-b737-4bde6e19acff admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:00.417943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad13ac49bc9b4776bc85774526b25124', 'request_id': 'req-3622bd12-692b-466d-9b89-f96eebb6fa7a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:00.447505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:00.494916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 947/1877] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:00 2022] POST /identity/v3/projects => generated 389 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:53:00.495937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a621-c29e-4503-b737-4bde6e19acff admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:00.525821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad13ac49bc9b4776bc85774526b25124', 'request_id': 'req-3622bd12-692b-466d-9b89-f96eebb6fa7a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:00.533983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad13ac49bc9b4776bc85774526b25124', 'request_id': 'req-3622bd12-692b-466d-9b89-f96eebb6fa7a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:00.552933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] The token cache is being invalidate because project ad13ac49bc9b4776bc85774526b25124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:00.553507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:00.555275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3622bd12-692b-466d-9b89-f96eebb6fa7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:00.561986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 931/1878] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:00 2022] DELETE /identity/v3/projects/ad13ac49bc9b4776bc85774526b25124 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:00.565052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0fae3-a0d4-48ee-86ee-7eef91b78fdf None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.565347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0fae3-a0d4-48ee-86ee-7eef91b78fdf None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.565631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0fae3-a0d4-48ee-86ee-7eef91b78fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.589735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a621-c29e-4503-b737-4bde6e19acff admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:00.590288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c8c0fae3-a0d4-48ee-86ee-7eef91b78fdf None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:00.591633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c8c0fae3-a0d4-48ee-86ee-7eef91b78fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:00.595987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2badd3-89b4-4218-9b48-16896359a83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:00.596896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2badd3-89b4-4218-9b48-16896359a83b admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.597559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2badd3-89b4-4218-9b48-16896359a83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.598050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2badd3-89b4-4218-9b48-16896359a83b admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.599040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2badd3-89b4-4218-9b48-16896359a83b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:00.601452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2badd3-89b4-4218-9b48-16896359a83b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:00.603590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 932/1879] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:53:00.614372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81745db6-118d-4fde-9a0d-bfea730289bc None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.615021 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81745db6-118d-4fde-9a0d-bfea730289bc None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.615021 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81745db6-118d-4fde-9a0d-bfea730289bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.641718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-81745db6-118d-4fde-9a0d-bfea730289bc None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:00.642822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-81745db6-118d-4fde-9a0d-bfea730289bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:00.664335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 933/1880] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:53:00.669723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81745db6-118d-4fde-9a0d-bfea730289bc None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:00.684198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81745db6-118d-4fde-9a0d-bfea730289bc None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:00.692071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd39cff-7922-4e23-a07f-5c94d9554cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:00.692707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39cff-7922-4e23-a07f-5c94d9554cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.692994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39cff-7922-4e23-a07f-5c94d9554cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.693276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39cff-7922-4e23-a07f-5c94d9554cbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.693959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd39cff-7922-4e23-a07f-5c94d9554cbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:53:00.694268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd39cff-7922-4e23-a07f-5c94d9554cbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:00.695971 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd39cff-7922-4e23-a07f-5c94d9554cbc admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:00.715383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 934/1881] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:53:00.721056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd39cff-7922-4e23-a07f-5c94d9554cbc admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:00.734062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd39cff-7922-4e23-a07f-5c94d9554cbc admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:00.741565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433a85b-0afa-401e-a5ae-41f2345b549d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:00.742563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433a85b-0afa-401e-a5ae-41f2345b549d admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.742712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433a85b-0afa-401e-a5ae-41f2345b549d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.742935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433a85b-0afa-401e-a5ae-41f2345b549d admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.743580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433a85b-0afa-401e-a5ae-41f2345b549d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:00.745542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433a85b-0afa-401e-a5ae-41f2345b549d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:00.772034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 948/1882] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:00 2022] POST /identity/v3/users => generated 301 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:00.774533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2badd3-89b4-4218-9b48-16896359a83b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:00.775587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-de2badd3-89b4-4218-9b48-16896359a83b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:00.789677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2badd3-89b4-4218-9b48-16896359a83b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:00.796120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb3a6-e272-4029-a9ef-523e949980e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:00.796936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb3a6-e272-4029-a9ef-523e949980e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.797319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb3a6-e272-4029-a9ef-523e949980e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.797837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fb3a6-e272-4029-a9ef-523e949980e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.799554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fb3a6-e272-4029-a9ef-523e949980e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:00.803279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fb3a6-e272-4029-a9ef-523e949980e4 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:00.818542 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 949/1883] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:53:00.821272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb3a6-e272-4029-a9ef-523e949980e4 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:00.834680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fb3a6-e272-4029-a9ef-523e949980e4 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:00.840266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e6ac-43d4-4e13-b2eb-58396898e236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:00.840971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e6ac-43d4-4e13-b2eb-58396898e236 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.841257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e6ac-43d4-4e13-b2eb-58396898e236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.841537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e6ac-43d4-4e13-b2eb-58396898e236 admin admin] PATH_INFO: `/v3/projects/9d8c711ef6684c3eb16a212877155d49/users/f17bad135ebe4cbd908f439f59a0f020/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:00.866005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2e6ac-43d4-4e13-b2eb-58396898e236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8c711ef6684c3eb16a212877155d49, user_id=f17bad135ebe4cbd908f439f59a0f020, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:00.869107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2e6ac-43d4-4e13-b2eb-58396898e236 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:00.899814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 935/1884] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:00 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:00.903807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433a85b-0afa-401e-a5ae-41f2345b549d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:00.991918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433a85b-0afa-401e-a5ae-41f2345b549d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:00.996829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf3e24-e2d7-4623-be26-2e9da5c6058f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:00.999094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf3e24-e2d7-4623-be26-2e9da5c6058f admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:00.999395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf3e24-e2d7-4623-be26-2e9da5c6058f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:00.999866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf3e24-e2d7-4623-be26-2e9da5c6058f admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:01.005720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf3e24-e2d7-4623-be26-2e9da5c6058f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:01.005720 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf3e24-e2d7-4623-be26-2e9da5c6058f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:01.046801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 950/1885] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:00 2022] PUT /identity/v3/projects/9d8c711ef6684c3eb16a212877155d49/users/f17bad135ebe4cbd908f439f59a0f020/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:01.051282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e6ac-43d4-4e13-b2eb-58396898e236 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:01.155639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 936/1886] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:00 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:53:01.158195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf3e24-e2d7-4623-be26-2e9da5c6058f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:01.192890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf3e24-e2d7-4623-be26-2e9da5c6058f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:01.208206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8e0a9-987c-4071-8caa-c94bb31d6cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:01.210977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8e0a9-987c-4071-8caa-c94bb31d6cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:01.212531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8e0a9-987c-4071-8caa-c94bb31d6cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:01.213094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8e0a9-987c-4071-8caa-c94bb31d6cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:01.219891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8e0a9-987c-4071-8caa-c94bb31d6cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:01.220011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e6ac-43d4-4e13-b2eb-58396898e236 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:01.224459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8e0a9-987c-4071-8caa-c94bb31d6cfc admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:01.224674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c516c-bf4e-4d4b-b2f4-cef5ed1dc460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:01.225178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c516c-bf4e-4d4b-b2f4-cef5ed1dc460 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:01.225587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c516c-bf4e-4d4b-b2f4-cef5ed1dc460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:01.226008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c516c-bf4e-4d4b-b2f4-cef5ed1dc460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:01.227522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c516c-bf4e-4d4b-b2f4-cef5ed1dc460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:01.230198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c516c-bf4e-4d4b-b2f4-cef5ed1dc460 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:01.241659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 951/1887] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:01.245939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c516c-bf4e-4d4b-b2f4-cef5ed1dc460 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:01.256580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 937/1888] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:01.261631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c516c-bf4e-4d4b-b2f4-cef5ed1dc460 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:01.264420 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8e0a9-987c-4071-8caa-c94bb31d6cfc admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:01.275383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72c100-63bf-4eb5-acd1-6e3664624fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:01.276407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72c100-63bf-4eb5-acd1-6e3664624fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:01.277060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72c100-63bf-4eb5-acd1-6e3664624fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:01.278610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72c100-63bf-4eb5-acd1-6e3664624fc1 admin admin] PATH_INFO: `/v3/projects/9d8c711ef6684c3eb16a212877155d49/users/f17bad135ebe4cbd908f439f59a0f020/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:01.293239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72c100-63bf-4eb5-acd1-6e3664624fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8c711ef6684c3eb16a212877155d49, user_id=f17bad135ebe4cbd908f439f59a0f020, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:01.295562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:01.301357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72c100-63bf-4eb5-acd1-6e3664624fc1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:01.325255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 952/1889] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:01 2022] PUT /identity/v3/projects/9d8c711ef6684c3eb16a212877155d49/users/f17bad135ebe4cbd908f439f59a0f020/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:01.327173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438b1b5-0660-460a-b943-20f36ac4f1df None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:01.327567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438b1b5-0660-460a-b943-20f36ac4f1df None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:01.327949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438b1b5-0660-460a-b943-20f36ac4f1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:01.338438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8e0a9-987c-4071-8caa-c94bb31d6cfc admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:01.366312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-1438b1b5-0660-460a-b943-20f36ac4f1df None None] MFA Rules not processed for user `f17bad135ebe4cbd908f439f59a0f020`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:01.368376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1438b1b5-0660-460a-b943-20f36ac4f1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:01.497247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 953/1890] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:01 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:01.518908 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d98b23-efd3-438c-a154-1ad11796f430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:01.519680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d98b23-efd3-438c-a154-1ad11796f430 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:01.520032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d98b23-efd3-438c-a154-1ad11796f430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:01.520375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d98b23-efd3-438c-a154-1ad11796f430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:01.521177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d98b23-efd3-438c-a154-1ad11796f430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:01.523676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438b1b5-0660-460a-b943-20f36ac4f1df None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:01.524870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d98b23-efd3-438c-a154-1ad11796f430 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:01.525286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1438b1b5-0660-460a-b943-20f36ac4f1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:01.537980 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 938/1891] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:01.543982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d98b23-efd3-438c-a154-1ad11796f430 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:01.559225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d98b23-efd3-438c-a154-1ad11796f430 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:01.580402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac4312-297f-4d71-9e1b-cc87bcd12f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:01.581341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac4312-297f-4d71-9e1b-cc87bcd12f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:01.581707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac4312-297f-4d71-9e1b-cc87bcd12f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:01.582198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac4312-297f-4d71-9e1b-cc87bcd12f45 admin admin] PATH_INFO: `/v3/projects/8e50d5d8bb7b4b37816a4c06688bc99f/users/46fd101cf7bd427e85c57a0e78e3be85/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:01.603729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac4312-297f-4d71-9e1b-cc87bcd12f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e50d5d8bb7b4b37816a4c06688bc99f, user_id=46fd101cf7bd427e85c57a0e78e3be85, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:01.606619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac4312-297f-4d71-9e1b-cc87bcd12f45 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:01.617385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438b1b5-0660-460a-b943-20f36ac4f1df None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:01.626151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbede8ce-9c31-48e8-b7b7-3adc87169ab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:01.626918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbede8ce-9c31-48e8-b7b7-3adc87169ab9 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:01.627265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbede8ce-9c31-48e8-b7b7-3adc87169ab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:01.627617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbede8ce-9c31-48e8-b7b7-3adc87169ab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:01.638155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbede8ce-9c31-48e8-b7b7-3adc87169ab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:01.640648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbede8ce-9c31-48e8-b7b7-3adc87169ab9 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:01.653407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 954/1892] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:01 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:01.753542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 939/1893] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:01 2022] PUT /identity/v3/projects/8e50d5d8bb7b4b37816a4c06688bc99f/users/46fd101cf7bd427e85c57a0e78e3be85/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:01.763010 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9191a-065d-40d3-a3a2-0e8ab8770c45 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:01.763436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9191a-065d-40d3-a3a2-0e8ab8770c45 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:01.763832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9191a-065d-40d3-a3a2-0e8ab8770c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:01.817094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-8bd9191a-065d-40d3-a3a2-0e8ab8770c45 None None] MFA Rules not processed for user `46fd101cf7bd427e85c57a0e78e3be85`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:01.819067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8bd9191a-065d-40d3-a3a2-0e8ab8770c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:01.839568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac4312-297f-4d71-9e1b-cc87bcd12f45 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:01.919212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 955/1894] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:01.925298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9191a-065d-40d3-a3a2-0e8ab8770c45 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:02.003469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac4312-297f-4d71-9e1b-cc87bcd12f45 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:02.014114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:02.014876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:02.015163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:02.015447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] PATH_INFO: `/v3/users/67beb33068e54bd8a98da10b42fcdae3` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:02.018315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67beb33068e54bd8a98da10b42fcdae3)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:02.023056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:02.048832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9191a-065d-40d3-a3a2-0e8ab8770c45 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:02.056955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4812718-e8cb-43ca-af9a-c097f526d6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:02.057567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4812718-e8cb-43ca-af9a-c097f526d6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:02.057853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4812718-e8cb-43ca-af9a-c097f526d6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:02.059069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4812718-e8cb-43ca-af9a-c097f526d6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:02.059608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4812718-e8cb-43ca-af9a-c097f526d6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:02.063189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4812718-e8cb-43ca-af9a-c097f526d6f1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:02.282173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 956/1895] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:01 2022] POST /identity/v3/projects => generated 387 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:02.287020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4812718-e8cb-43ca-af9a-c097f526d6f1 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:02.404681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4812718-e8cb-43ca-af9a-c097f526d6f1 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:02.415597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02505e81-8337-4d7e-b079-725ecda7ca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:02.418962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02505e81-8337-4d7e-b079-725ecda7ca0f admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:02.418962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02505e81-8337-4d7e-b079-725ecda7ca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:02.418962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02505e81-8337-4d7e-b079-725ecda7ca0f admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:02.419281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02505e81-8337-4d7e-b079-725ecda7ca0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:02.421635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02505e81-8337-4d7e-b079-725ecda7ca0f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:02.433646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67beb33068e54bd8a98da10b42fcdae3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:02.537819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67beb33068e54bd8a98da10b42fcdae3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:02.559032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67beb33068e54bd8a98da10b42fcdae3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:02.567096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67beb33068e54bd8a98da10b42fcdae3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:02.568030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f7166d85-c320-4a26-805a-dc1822a5c0f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67beb33068e54bd8a98da10b42fcdae3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:02.575667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 940/1896] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:01 2022] DELETE /identity/v3/users/67beb33068e54bd8a98da10b42fcdae3 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:02.582082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303abc12-dd85-4906-ac3d-61783a598400 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:02.582372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303abc12-dd85-4906-ac3d-61783a598400 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:02.583204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303abc12-dd85-4906-ac3d-61783a598400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:02.588686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 957/1897] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:02 2022] POST /identity/v3/users => generated 300 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:53:02.591039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02505e81-8337-4d7e-b079-725ecda7ca0f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:02.596907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-02505e81-8337-4d7e-b079-725ecda7ca0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:02.621574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-303abc12-dd85-4906-ac3d-61783a598400 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:02.623069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02505e81-8337-4d7e-b079-725ecda7ca0f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:02.623213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-303abc12-dd85-4906-ac3d-61783a598400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:02.710904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b5617-2ce1-4e61-acb8-aecb0caa8f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:02.714343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 941/1898] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:53:02.714482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b5617-2ce1-4e61-acb8-aecb0caa8f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:02.714482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b5617-2ce1-4e61-acb8-aecb0caa8f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:02.714482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b5617-2ce1-4e61-acb8-aecb0caa8f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:02.714482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b5617-2ce1-4e61-acb8-aecb0caa8f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:02.715310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b5617-2ce1-4e61-acb8-aecb0caa8f6f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:02.722327 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303abc12-dd85-4906-ac3d-61783a598400 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:02.724042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-303abc12-dd85-4906-ac3d-61783a598400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:02.728071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 958/1899] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:02.739957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b5617-2ce1-4e61-acb8-aecb0caa8f6f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:02.750600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b5617-2ce1-4e61-acb8-aecb0caa8f6f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:02.756307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0066285-127b-467d-b60d-07f644b71d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:02.757071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0066285-127b-467d-b60d-07f644b71d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:02.757425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0066285-127b-467d-b60d-07f644b71d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:02.757807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0066285-127b-467d-b60d-07f644b71d3e admin admin] PATH_INFO: `/v3/projects/4c93259d3f9b4189a8321840649a7b91/users/c821e1ade82d42bebb6985f2a09034af/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:02.775703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0066285-127b-467d-b60d-07f644b71d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c93259d3f9b4189a8321840649a7b91, user_id=c821e1ade82d42bebb6985f2a09034af, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:02.778627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0066285-127b-467d-b60d-07f644b71d3e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:02.804128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303abc12-dd85-4906-ac3d-61783a598400 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:02.807744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87b6ac-460d-4789-b422-b295ef19b298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:02.808356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87b6ac-460d-4789-b422-b295ef19b298 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:02.808659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87b6ac-460d-4789-b422-b295ef19b298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:02.808960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87b6ac-460d-4789-b422-b295ef19b298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:02.823311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87b6ac-460d-4789-b422-b295ef19b298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:02.823311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87b6ac-460d-4789-b422-b295ef19b298 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:02.835508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 942/1900] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:53:02.893638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 959/1901] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:02 2022] PUT /identity/v3/projects/4c93259d3f9b4189a8321840649a7b91/users/c821e1ade82d42bebb6985f2a09034af/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:02.898092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87b6ac-460d-4789-b422-b295ef19b298 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:02.910191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87b6ac-460d-4789-b422-b295ef19b298 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:02.973258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31368b5b-e691-454c-a0f6-65ee8e34299e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:02.973883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31368b5b-e691-454c-a0f6-65ee8e34299e admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:02.974165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31368b5b-e691-454c-a0f6-65ee8e34299e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:02.974437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31368b5b-e691-454c-a0f6-65ee8e34299e admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:02.975119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31368b5b-e691-454c-a0f6-65ee8e34299e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:02.976868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31368b5b-e691-454c-a0f6-65ee8e34299e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:02.987553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 943/1902] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:02.992654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0066285-127b-467d-b60d-07f644b71d3e admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:03.003367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0066285-127b-467d-b60d-07f644b71d3e admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:03.008913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281fdc6-ccf3-4af8-b375-2069f360be03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:03.009694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281fdc6-ccf3-4af8-b375-2069f360be03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:03.010044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281fdc6-ccf3-4af8-b375-2069f360be03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:03.010389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281fdc6-ccf3-4af8-b375-2069f360be03 admin admin] PATH_INFO: `/v3/projects/4c93259d3f9b4189a8321840649a7b91/users/c821e1ade82d42bebb6985f2a09034af/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:03.012451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281fdc6-ccf3-4af8-b375-2069f360be03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c93259d3f9b4189a8321840649a7b91, user_id=c821e1ade82d42bebb6985f2a09034af, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:03.015701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281fdc6-ccf3-4af8-b375-2069f360be03 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:03.034557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 960/1903] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:02 2022] PUT /identity/v3/projects/4c93259d3f9b4189a8321840649a7b91/users/c821e1ade82d42bebb6985f2a09034af/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:03.082791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb861a-5716-4d42-9677-b3b72618486b None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:03.083094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb861a-5716-4d42-9677-b3b72618486b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:03.083381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb861a-5716-4d42-9677-b3b72618486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:03.114096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:03.120475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-4dcb861a-5716-4d42-9677-b3b72618486b None None] MFA Rules not processed for user `c821e1ade82d42bebb6985f2a09034af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:03.125221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4dcb861a-5716-4d42-9677-b3b72618486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:03.217460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 944/1904] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:03.273044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281fdc6-ccf3-4af8-b375-2069f360be03 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:03.274850 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2281fdc6-ccf3-4af8-b375-2069f360be03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:03.351955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281fdc6-ccf3-4af8-b375-2069f360be03 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:03.356987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e152b-456f-4cd6-9bc5-ce9aa1699341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:03.358089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e152b-456f-4cd6-9bc5-ce9aa1699341 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:03.358089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e152b-456f-4cd6-9bc5-ce9aa1699341 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:03.358274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e152b-456f-4cd6-9bc5-ce9aa1699341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:03.374007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e152b-456f-4cd6-9bc5-ce9aa1699341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:03.375564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e152b-456f-4cd6-9bc5-ce9aa1699341 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:03.390653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 961/1905] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:03.555031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb861a-5716-4d42-9677-b3b72618486b None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:03.555031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4dcb861a-5716-4d42-9677-b3b72618486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:03.644282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb861a-5716-4d42-9677-b3b72618486b None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:03.649215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:03.649880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:03.650162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:03.650445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] PATH_INFO: `/v3/projects/5aad49c87c4449309f0947e70b486565` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:03.651646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aad49c87c4449309f0947e70b486565)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:03.653643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:03.776812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aad49c87c4449309f0947e70b486565', 'request_id': 'req-88399134-ae1c-4339-b733-623c62664598'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:03.863275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:03.909088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aad49c87c4449309f0947e70b486565', 'request_id': 'req-88399134-ae1c-4339-b733-623c62664598'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:03.915929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aad49c87c4449309f0947e70b486565', 'request_id': 'req-88399134-ae1c-4339-b733-623c62664598'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:03.928961 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] The token cache is being invalidate because project 5aad49c87c4449309f0947e70b486565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:03.928961 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:03.929149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:03.931492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 945/1906] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:03 2022] DELETE /identity/v3/projects/5aad49c87c4449309f0947e70b486565 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:03.936255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e152b-456f-4cd6-9bc5-ce9aa1699341 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:03.948466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e152b-456f-4cd6-9bc5-ce9aa1699341 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:04.012354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:04.013002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:04.013290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:04.013566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] PATH_INFO: `/v3/users/f17bad135ebe4cbd908f439f59a0f020` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:04.014738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f17bad135ebe4cbd908f439f59a0f020)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:04.017400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:04.357242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f17bad135ebe4cbd908f439f59a0f020'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:04.358076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f17bad135ebe4cbd908f439f59a0f020'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:04.434303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f17bad135ebe4cbd908f439f59a0f020'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:04.442796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f17bad135ebe4cbd908f439f59a0f020'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:04.454933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f17bad135ebe4cbd908f439f59a0f020'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:04.460875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 962/1907] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:03 2022] DELETE /identity/v3/users/f17bad135ebe4cbd908f439f59a0f020 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:05.049437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:05.051559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:05.066624 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88399134-ae1c-4339-b733-623c62664598 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:05.096105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:05.098171 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:05.146665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:05.147913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:05.148360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:05.148654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] PATH_INFO: `/v3/users/46fd101cf7bd427e85c57a0e78e3be85` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:05.149894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46fd101cf7bd427e85c57a0e78e3be85)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:05.152184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:05.196112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27174d4f-11f8-450b-a640-397b5e8b1bf3 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:05.201607 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:05.202347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:05.202734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:05.203515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] PATH_INFO: `/v3/projects/9d8c711ef6684c3eb16a212877155d49` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:05.206712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d8c711ef6684c3eb16a212877155d49)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:05.207089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:05.398310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46fd101cf7bd427e85c57a0e78e3be85'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.405944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:05.412683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d8c711ef6684c3eb16a212877155d49', 'request_id': 'req-105352e2-5c78-4457-a6c8-5db3e5dbf22e'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.418586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d8c711ef6684c3eb16a212877155d49', 'request_id': 'req-105352e2-5c78-4457-a6c8-5db3e5dbf22e'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.481077 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46fd101cf7bd427e85c57a0e78e3be85'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.497534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46fd101cf7bd427e85c57a0e78e3be85'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.507084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46fd101cf7bd427e85c57a0e78e3be85'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.508125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2d780d8-2d89-4e2f-9eec-6fd1f9261014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46fd101cf7bd427e85c57a0e78e3be85'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.515617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 946/1908] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:05 2022] DELETE /identity/v3/users/46fd101cf7bd427e85c57a0e78e3be85 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:05.522978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c868d3-f926-4f91-a7f7-3248e4e5f448 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:05.523269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c868d3-f926-4f91-a7f7-3248e4e5f448 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:05.523729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c868d3-f926-4f91-a7f7-3248e4e5f448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:05.543964 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-88c868d3-f926-4f91-a7f7-3248e4e5f448 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:05.545239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-88c868d3-f926-4f91-a7f7-3248e4e5f448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:05.593514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d8c711ef6684c3eb16a212877155d49', 'request_id': 'req-105352e2-5c78-4457-a6c8-5db3e5dbf22e'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.607468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] The token cache is being invalidate because project 9d8c711ef6684c3eb16a212877155d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:05.607868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.608938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:05.611742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 947/1909] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:05.612533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 963/1910] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:05 2022] DELETE /identity/v3/projects/9d8c711ef6684c3eb16a212877155d49 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:05.620338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958a260-b265-45da-a8ce-be6f539c779e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:05.620637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958a260-b265-45da-a8ce-be6f539c779e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:05.620912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958a260-b265-45da-a8ce-be6f539c779e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:05.626327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:05.628317 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:05.642912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2958a260-b265-45da-a8ce-be6f539c779e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:05.644171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2958a260-b265-45da-a8ce-be6f539c779e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:05.654892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 948/1911] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:05.664424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7babc3-7ceb-45e8-ac15-889af7864ac3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:05.664716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7babc3-7ceb-45e8-ac15-889af7864ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:05.664995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7babc3-7ceb-45e8-ac15-889af7864ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:05.686530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-8d7babc3-7ceb-45e8-ac15-889af7864ac3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:05.687809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8d7babc3-7ceb-45e8-ac15-889af7864ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:05.698938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 949/1912] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:53:05.703301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7babc3-7ceb-45e8-ac15-889af7864ac3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:05.703762 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105352e2-5c78-4457-a6c8-5db3e5dbf22e admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:05.713673 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7babc3-7ceb-45e8-ac15-889af7864ac3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:05.720013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ff678-6995-47f0-801b-b75226df4dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:05.720995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ff678-6995-47f0-801b-b75226df4dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:05.721224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ff678-6995-47f0-801b-b75226df4dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:05.721499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ff678-6995-47f0-801b-b75226df4dd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:05.722161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ff678-6995-47f0-801b-b75226df4dd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:53:05.722468 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ff678-6995-47f0-801b-b75226df4dd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:05.724553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ff678-6995-47f0-801b-b75226df4dd0 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:05.737814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 950/1913] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:53:05.743063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ff678-6995-47f0-801b-b75226df4dd0 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:05.757111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ff678-6995-47f0-801b-b75226df4dd0 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:05.762096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904399-5ea0-4dc2-8b55-34222664bef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:05.762902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904399-5ea0-4dc2-8b55-34222664bef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:05.763183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904399-5ea0-4dc2-8b55-34222664bef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:05.763636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904399-5ea0-4dc2-8b55-34222664bef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:05.764273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8904399-5ea0-4dc2-8b55-34222664bef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:05.767609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b942a-0702-47e5-bf0d-460f86334629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:05.767787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8904399-5ea0-4dc2-8b55-34222664bef5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:05.767871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b942a-0702-47e5-bf0d-460f86334629 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:05.767871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b942a-0702-47e5-bf0d-460f86334629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:05.767871 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b942a-0702-47e5-bf0d-460f86334629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:05.768911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-415b942a-0702-47e5-bf0d-460f86334629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:05.781206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b942a-0702-47e5-bf0d-460f86334629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:05.782738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b942a-0702-47e5-bf0d-460f86334629 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:05.793119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 964/1914] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:05.867290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 951/1915] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:05 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:05.871022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b942a-0702-47e5-bf0d-460f86334629 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:05.881779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b942a-0702-47e5-bf0d-460f86334629 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:05.951307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996441bb-ed1a-4fe1-917f-7c9885900d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:05.952031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996441bb-ed1a-4fe1-917f-7c9885900d38 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:05.952323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996441bb-ed1a-4fe1-917f-7c9885900d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:05.952599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996441bb-ed1a-4fe1-917f-7c9885900d38 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:05.953245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996441bb-ed1a-4fe1-917f-7c9885900d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:05.955429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996441bb-ed1a-4fe1-917f-7c9885900d38 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:06.154933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 965/1916] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:05 2022] POST /identity/v3/users => generated 289 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:53:06.159729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904399-5ea0-4dc2-8b55-34222664bef5 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:06.173346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904399-5ea0-4dc2-8b55-34222664bef5 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:06.184785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584ffa9-d025-4583-9ac7-581f27212699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:06.185577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584ffa9-d025-4583-9ac7-581f27212699 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:06.191513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584ffa9-d025-4583-9ac7-581f27212699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:06.191513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584ffa9-d025-4583-9ac7-581f27212699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:06.191513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584ffa9-d025-4583-9ac7-581f27212699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:06.191513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584ffa9-d025-4583-9ac7-581f27212699 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:06.210058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 952/1917] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:06.213947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996441bb-ed1a-4fe1-917f-7c9885900d38 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:06.225419 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996441bb-ed1a-4fe1-917f-7c9885900d38 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:06.234313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fbedc-7d93-4ff2-8dcb-c5616a910066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:06.235875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fbedc-7d93-4ff2-8dcb-c5616a910066 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:06.236378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fbedc-7d93-4ff2-8dcb-c5616a910066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:06.237410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fbedc-7d93-4ff2-8dcb-c5616a910066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:06.238068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fbedc-7d93-4ff2-8dcb-c5616a910066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:06.240924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fbedc-7d93-4ff2-8dcb-c5616a910066 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:06.259787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 966/1918] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:06.263281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584ffa9-d025-4583-9ac7-581f27212699 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:06.275524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584ffa9-d025-4583-9ac7-581f27212699 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:06.281635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142a33b-9423-4505-b4b2-f3f83b595cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:06.282570 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142a33b-9423-4505-b4b2-f3f83b595cae admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:06.283115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142a33b-9423-4505-b4b2-f3f83b595cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:06.283621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142a33b-9423-4505-b4b2-f3f83b595cae admin admin] PATH_INFO: `/v3/projects/73ca51287a13466b93b8fe176a752e75/users/c07f736d7705478885919532338cc5a9/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:06.314656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142a33b-9423-4505-b4b2-f3f83b595cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ca51287a13466b93b8fe176a752e75, user_id=c07f736d7705478885919532338cc5a9, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:06.317535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142a33b-9423-4505-b4b2-f3f83b595cae admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:06.443370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 953/1919] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:06 2022] PUT /identity/v3/projects/73ca51287a13466b93b8fe176a752e75/users/c07f736d7705478885919532338cc5a9/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:06.454215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccd6f7-09eb-4b00-b0f3-0ebb890ce953 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:06.454550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccd6f7-09eb-4b00-b0f3-0ebb890ce953 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:06.454806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccd6f7-09eb-4b00-b0f3-0ebb890ce953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:06.478410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142a33b-9423-4505-b4b2-f3f83b595cae admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:06.480360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9142a33b-9423-4505-b4b2-f3f83b595cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:06.494020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-99ccd6f7-09eb-4b00-b0f3-0ebb890ce953 None None] MFA Rules not processed for user `c07f736d7705478885919532338cc5a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:06.498588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-99ccd6f7-09eb-4b00-b0f3-0ebb890ce953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:06.556628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:06.591844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142a33b-9423-4505-b4b2-f3f83b595cae admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:06.595386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 967/1920] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:06.598939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:06.599932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:06.600360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:06.600738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] PATH_INFO: `/v3/projects/8e50d5d8bb7b4b37816a4c06688bc99f` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:06.602917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e50d5d8bb7b4b37816a4c06688bc99f)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:06.608821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:06.613078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccd6f7-09eb-4b00-b0f3-0ebb890ce953 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:06.614965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-99ccd6f7-09eb-4b00-b0f3-0ebb890ce953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:06.718968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccd6f7-09eb-4b00-b0f3-0ebb890ce953 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:06.723369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4d937-ddea-43d0-8596-99e1ac919812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:06.723369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4d937-ddea-43d0-8596-99e1ac919812 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:06.723792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4d937-ddea-43d0-8596-99e1ac919812 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:06.724074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4d937-ddea-43d0-8596-99e1ac919812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:06.748287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4d937-ddea-43d0-8596-99e1ac919812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:06.752882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4d937-ddea-43d0-8596-99e1ac919812 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:06.762315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 968/1921] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:53:06.772799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e50d5d8bb7b4b37816a4c06688bc99f', 'request_id': 'req-723a2aac-ede9-46c3-bc2b-45f437834f14'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:06.884910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e50d5d8bb7b4b37816a4c06688bc99f', 'request_id': 'req-723a2aac-ede9-46c3-bc2b-45f437834f14'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:06.899545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e50d5d8bb7b4b37816a4c06688bc99f', 'request_id': 'req-723a2aac-ede9-46c3-bc2b-45f437834f14'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:06.924082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] The token cache is being invalidate because project 8e50d5d8bb7b4b37816a4c06688bc99f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:06.924449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:06.925175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:06.930555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 954/1922] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:06 2022] DELETE /identity/v3/projects/8e50d5d8bb7b4b37816a4c06688bc99f => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:06.932519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4d937-ddea-43d0-8596-99e1ac919812 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:06.960252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4d937-ddea-43d0-8596-99e1ac919812 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:07.056091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:07.056866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:07.057196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:07.057477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] PATH_INFO: `/v3/users/c821e1ade82d42bebb6985f2a09034af` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:07.058749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c821e1ade82d42bebb6985f2a09034af)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:07.064147 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:07.445248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c821e1ade82d42bebb6985f2a09034af'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:07.446269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c821e1ade82d42bebb6985f2a09034af'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:07.513894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c821e1ade82d42bebb6985f2a09034af'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:07.529735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c821e1ade82d42bebb6985f2a09034af'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:07.550718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f35e41a5-96e2-4758-af63-666204a264ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c821e1ade82d42bebb6985f2a09034af'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:07.557554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 969/1923] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:06 2022] DELETE /identity/v3/users/c821e1ade82d42bebb6985f2a09034af => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:08.425110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:08.426758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:08.472297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a2aac-ede9-46c3-bc2b-45f437834f14 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:08.546625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:08.547716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:08.548014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:08.548292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] PATH_INFO: `/v3/projects/4c93259d3f9b4189a8321840649a7b91` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:08.549558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c93259d3f9b4189a8321840649a7b91)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:08.551689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:08.709135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c93259d3f9b4189a8321840649a7b91', 'request_id': 'req-d8bbccde-709d-4346-a26d-cfc6e195fd5a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:08.822027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c93259d3f9b4189a8321840649a7b91', 'request_id': 'req-d8bbccde-709d-4346-a26d-cfc6e195fd5a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:08.829541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c93259d3f9b4189a8321840649a7b91', 'request_id': 'req-d8bbccde-709d-4346-a26d-cfc6e195fd5a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:08.842691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] The token cache is being invalidate because project 4c93259d3f9b4189a8321840649a7b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:08.854943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:08.855768 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d8bbccde-709d-4346-a26d-cfc6e195fd5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:08.857737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 955/1924] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:08 2022] DELETE /identity/v3/projects/4c93259d3f9b4189a8321840649a7b91 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:08.863242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263872b-172a-4676-81e6-4b1abb587f57 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:08.863764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263872b-172a-4676-81e6-4b1abb587f57 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:08.863764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263872b-172a-4676-81e6-4b1abb587f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:08.903956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-3263872b-172a-4676-81e6-4b1abb587f57 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:08.905204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3263872b-172a-4676-81e6-4b1abb587f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:09.025548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 970/1925] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:09.025791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac5f2f-1e30-4302-89bb-72c5271615e7 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:09.026079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac5f2f-1e30-4302-89bb-72c5271615e7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:09.026360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac5f2f-1e30-4302-89bb-72c5271615e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:09.069510 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-a1ac5f2f-1e30-4302-89bb-72c5271615e7 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:09.071429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a1ac5f2f-1e30-4302-89bb-72c5271615e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:09.089033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 956/1926] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:09.090499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263872b-172a-4676-81e6-4b1abb587f57 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:09.101808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263872b-172a-4676-81e6-4b1abb587f57 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:09.106932 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a9f2c-7808-4922-bde9-4d3cdb487c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:09.107574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a9f2c-7808-4922-bde9-4d3cdb487c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:09.107892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a9f2c-7808-4922-bde9-4d3cdb487c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:09.108561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a9f2c-7808-4922-bde9-4d3cdb487c32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:09.108800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a9f2c-7808-4922-bde9-4d3cdb487c32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:53:09.110312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a9f2c-7808-4922-bde9-4d3cdb487c32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:09.113049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a9f2c-7808-4922-bde9-4d3cdb487c32 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:09.131222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 971/1927] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:53:09.132516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac5f2f-1e30-4302-89bb-72c5271615e7 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:09.145703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac5f2f-1e30-4302-89bb-72c5271615e7 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:09.151353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc624c0-dd4b-4019-a565-6b6482dc2120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:09.152122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc624c0-dd4b-4019-a565-6b6482dc2120 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:09.152529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc624c0-dd4b-4019-a565-6b6482dc2120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:09.152909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc624c0-dd4b-4019-a565-6b6482dc2120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:09.153957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc624c0-dd4b-4019-a565-6b6482dc2120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:09.156569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc624c0-dd4b-4019-a565-6b6482dc2120 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:09.299374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 957/1928] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:09 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:09.303390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a9f2c-7808-4922-bde9-4d3cdb487c32 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:09.316039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a9f2c-7808-4922-bde9-4d3cdb487c32 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:09.411786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c65b67-d02b-418e-93b8-ce25660a8e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:09.412416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c65b67-d02b-418e-93b8-ce25660a8e99 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:09.412704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c65b67-d02b-418e-93b8-ce25660a8e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:09.413142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c65b67-d02b-418e-93b8-ce25660a8e99 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:09.413798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c65b67-d02b-418e-93b8-ce25660a8e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:09.415909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c65b67-d02b-418e-93b8-ce25660a8e99 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:09.429135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:09.546051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 972/1929] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:09 2022] POST /identity/v3/users => generated 288 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:09.554252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc624c0-dd4b-4019-a565-6b6482dc2120 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:09.570926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc624c0-dd4b-4019-a565-6b6482dc2120 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:09.580490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64a92c-9ff8-4632-b262-4a179b187d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:09.581316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a92c-9ff8-4632-b262-4a179b187d54 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:09.581893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a92c-9ff8-4632-b262-4a179b187d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:09.582356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a92c-9ff8-4632-b262-4a179b187d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:09.583730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64a92c-9ff8-4632-b262-4a179b187d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:09.586204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64a92c-9ff8-4632-b262-4a179b187d54 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:09.598463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 958/1930] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:09.603619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c65b67-d02b-418e-93b8-ce25660a8e99 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:09.617892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c65b67-d02b-418e-93b8-ce25660a8e99 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:09.623883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8172581-6fa9-45f2-ae79-c010274222ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:09.625499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8172581-6fa9-45f2-ae79-c010274222ef admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:09.625800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8172581-6fa9-45f2-ae79-c010274222ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:09.626064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8172581-6fa9-45f2-ae79-c010274222ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:09.627345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8172581-6fa9-45f2-ae79-c010274222ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:09.629736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8172581-6fa9-45f2-ae79-c010274222ef admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:09.645031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 973/1931] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:53:09.649363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64a92c-9ff8-4632-b262-4a179b187d54 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:09.660036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64a92c-9ff8-4632-b262-4a179b187d54 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:09.664861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d301-862b-40c9-9e94-9b2e47b2e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:09.665521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d301-862b-40c9-9e94-9b2e47b2e0aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:09.665803 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d301-862b-40c9-9e94-9b2e47b2e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:09.669054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4d301-862b-40c9-9e94-9b2e47b2e0aa admin admin] PATH_INFO: `/v3/projects/82086ddb29eb4893b771c71b725d930a/users/35ce0f9dad99470cb666e685021216b2/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:09.683039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4d301-862b-40c9-9e94-9b2e47b2e0aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=82086ddb29eb4893b771c71b725d930a, user_id=35ce0f9dad99470cb666e685021216b2, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:09.696697 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4d301-862b-40c9-9e94-9b2e47b2e0aa admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:09.785690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 959/1932] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:09 2022] PUT /identity/v3/projects/82086ddb29eb4893b771c71b725d930a/users/35ce0f9dad99470cb666e685021216b2/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:09.794882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c87dd3-7083-4159-a406-75db60e7349d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:09.795175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c87dd3-7083-4159-a406-75db60e7349d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:09.795456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c87dd3-7083-4159-a406-75db60e7349d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:09.833798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-35c87dd3-7083-4159-a406-75db60e7349d None None] MFA Rules not processed for user `35ce0f9dad99470cb666e685021216b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:09.835054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-35c87dd3-7083-4159-a406-75db60e7349d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:09.910025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 974/1933] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:09.917435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d301-862b-40c9-9e94-9b2e47b2e0aa admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:09.919294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3fe4d301-862b-40c9-9e94-9b2e47b2e0aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:10.007610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4d301-862b-40c9-9e94-9b2e47b2e0aa admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:10.020940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9e699-9e8e-4fbb-a364-b486f09469ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ff3f53ddbb04eb8916b9ccfdb03dd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:10.021599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9e699-9e8e-4fbb-a364-b486f09469ef service glance] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:10.021884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9e699-9e8e-4fbb-a364-b486f09469ef service glance] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:10.022163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9e699-9e8e-4fbb-a364-b486f09469ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:10.040346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:10.048947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9e699-9e8e-4fbb-a364-b486f09469ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:10.050546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9e699-9e8e-4fbb-a364-b486f09469ef service glance] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:10.065075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 960/1934] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:11.427424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:11.507769 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:11.724598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:12.244407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:12.821359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:12.890009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:13.439924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:13.484127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:14.114133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:14.215621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:14.364132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:14.849576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:14.935595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:15.667206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:16.101125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:16.809704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:16.872740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:17.398689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:17.495313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:17.651433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:17.946594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:18.206147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:18.238961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:18.404066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:18.455099 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:18.592493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:18.786061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:18.894087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:18.962824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:28.695113 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:29.089328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:42.385547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdfbbb-2a06-4ec3-b310-ebd1827061d7 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:42.386696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdfbbb-2a06-4ec3-b310-ebd1827061d7 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:42.387181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdfbbb-2a06-4ec3-b310-ebd1827061d7 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:42.388768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 975/1935] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:53:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:53:42.401437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1d6df-1460-4986-a765-8dd023d3a6a3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:42.401964 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1d6df-1460-4986-a765-8dd023d3a6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:42.402148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1d6df-1460-4986-a765-8dd023d3a6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:42.423743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c6d1d6df-1460-4986-a765-8dd023d3a6a3 None None] MFA Rules not processed for user `cff23ae056ff452588cd0b6520e96962`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:42.424976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c6d1d6df-1460-4986-a765-8dd023d3a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:42.495771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 961/1936] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:53:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:42.520627 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1d6df-1460-4986-a765-8dd023d3a6a3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:42.524348 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdfbbb-2a06-4ec3-b310-ebd1827061d7 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:42.524728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c6d1d6df-1460-4986-a765-8dd023d3a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:42.526146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b6cdfbbb-2a06-4ec3-b310-ebd1827061d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:42.552236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdfbbb-2a06-4ec3-b310-ebd1827061d7 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:42.559828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc19ab1-f1cf-4b17-bd2f-f8283dbdc34e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:42.560797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc19ab1-f1cf-4b17-bd2f-f8283dbdc34e service placement] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:42.561328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc19ab1-f1cf-4b17-bd2f-f8283dbdc34e service placement] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:42.561787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc19ab1-f1cf-4b17-bd2f-f8283dbdc34e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:42.573513 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc19ab1-f1cf-4b17-bd2f-f8283dbdc34e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:42.575194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc19ab1-f1cf-4b17-bd2f-f8283dbdc34e service placement] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:42.585532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 976/1937] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:42.617596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1d6df-1460-4986-a765-8dd023d3a6a3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:42.622672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:42.623319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:42.623602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:42.623880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] PATH_INFO: `/v3/users/a466f40eba2a4975bd13614474ff527c` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:42.625057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a466f40eba2a4975bd13614474ff527c)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:42.627012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:42.889648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a466f40eba2a4975bd13614474ff527c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:42.987805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a466f40eba2a4975bd13614474ff527c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.001553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a466f40eba2a4975bd13614474ff527c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.009643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a466f40eba2a4975bd13614474ff527c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.010778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2202062b-5507-4510-9046-da65f933b23c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a466f40eba2a4975bd13614474ff527c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.017643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 962/1938] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:42 2022] DELETE /identity/v3/users/a466f40eba2a4975bd13614474ff527c => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:43.286425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc19ab1-f1cf-4b17-bd2f-f8283dbdc34e service placement] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.293997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc19ab1-f1cf-4b17-bd2f-f8283dbdc34e service placement] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.364526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.365178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.365463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.365742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] PATH_INFO: `/v3/projects/649f222d62844debb70df5ed5e8d44ad` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.367554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=649f222d62844debb70df5ed5e8d44ad)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.370182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.527532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '649f222d62844debb70df5ed5e8d44ad', 'request_id': 'req-6a6fe9a9-724d-417c-b960-4908ee31c114'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.533361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '649f222d62844debb70df5ed5e8d44ad', 'request_id': 'req-6a6fe9a9-724d-417c-b960-4908ee31c114'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.622474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '649f222d62844debb70df5ed5e8d44ad', 'request_id': 'req-6a6fe9a9-724d-417c-b960-4908ee31c114'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.635455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] The token cache is being invalidate because project 649f222d62844debb70df5ed5e8d44ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:43.635752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.638744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6a6fe9a9-724d-417c-b960-4908ee31c114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.638744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 977/1939] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:43 2022] DELETE /identity/v3/projects/649f222d62844debb70df5ed5e8d44ad => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:43.643619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20e710-a769-4692-9359-2b044d4e9368 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.644231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20e710-a769-4692-9359-2b044d4e9368 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.644231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20e710-a769-4692-9359-2b044d4e9368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.668312 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-5d20e710-a769-4692-9359-2b044d4e9368 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:43.669581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5d20e710-a769-4692-9359-2b044d4e9368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:43.679772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:43.732831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 963/1940] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:43.742084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c9f1e-7b62-4876-9443-acbb381fce5f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.742084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c9f1e-7b62-4876-9443-acbb381fce5f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.742084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c9f1e-7b62-4876-9443-acbb381fce5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.763078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6d6c9f1e-7b62-4876-9443-acbb381fce5f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:43.764381 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6d6c9f1e-7b62-4876-9443-acbb381fce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:43.772520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 978/1941] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:43.777375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20e710-a769-4692-9359-2b044d4e9368 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.787337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20e710-a769-4692-9359-2b044d4e9368 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.792339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3718-e92a-475b-b777-5f44ab41c512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.793216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3718-e92a-475b-b777-5f44ab41c512 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.793502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3718-e92a-475b-b777-5f44ab41c512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.793786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a3718-e92a-475b-b777-5f44ab41c512 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.794439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3718-e92a-475b-b777-5f44ab41c512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:53:43.794782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3718-e92a-475b-b777-5f44ab41c512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.796456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a3718-e92a-475b-b777-5f44ab41c512 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.808903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 964/1942] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:53:43.813263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c9f1e-7b62-4876-9443-acbb381fce5f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.823094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c9f1e-7b62-4876-9443-acbb381fce5f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.827643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f61cd5-2981-4f72-9afc-981cd20b99a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.828242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f61cd5-2981-4f72-9afc-981cd20b99a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.828523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f61cd5-2981-4f72-9afc-981cd20b99a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.831366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f61cd5-2981-4f72-9afc-981cd20b99a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.831366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f61cd5-2981-4f72-9afc-981cd20b99a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.831528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f61cd5-2981-4f72-9afc-981cd20b99a1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.947693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 979/1943] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:43.954801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3718-e92a-475b-b777-5f44ab41c512 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.966044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a3718-e92a-475b-b777-5f44ab41c512 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.030320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67639c78-ec1f-411e-a50a-fae77db7079c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.031251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67639c78-ec1f-411e-a50a-fae77db7079c admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.031251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67639c78-ec1f-411e-a50a-fae77db7079c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.031527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67639c78-ec1f-411e-a50a-fae77db7079c admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.032163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67639c78-ec1f-411e-a50a-fae77db7079c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.034098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67639c78-ec1f-411e-a50a-fae77db7079c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.162650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 965/1944] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/users => generated 282 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:53:44.165836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f61cd5-2981-4f72-9afc-981cd20b99a1 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.176790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f61cd5-2981-4f72-9afc-981cd20b99a1 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.185332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2a009-8fa8-48e6-9886-134c44d4134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.185332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2a009-8fa8-48e6-9886-134c44d4134b admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.185332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2a009-8fa8-48e6-9886-134c44d4134b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.185332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2a009-8fa8-48e6-9886-134c44d4134b admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.185707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2a009-8fa8-48e6-9886-134c44d4134b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.186169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2a009-8fa8-48e6-9886-134c44d4134b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.197303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 980/1945] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:44.202132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67639c78-ec1f-411e-a50a-fae77db7079c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.212333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67639c78-ec1f-411e-a50a-fae77db7079c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.216863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e388e2a-5679-4d79-8740-0dfadb77ade9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.217464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e388e2a-5679-4d79-8740-0dfadb77ade9 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.217750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e388e2a-5679-4d79-8740-0dfadb77ade9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.218031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e388e2a-5679-4d79-8740-0dfadb77ade9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.218726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e388e2a-5679-4d79-8740-0dfadb77ade9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.220347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e388e2a-5679-4d79-8740-0dfadb77ade9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.231245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 966/1946] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:44.235429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2a009-8fa8-48e6-9886-134c44d4134b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.245884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2a009-8fa8-48e6-9886-134c44d4134b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.250388 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686f46d-ab0c-47cf-9659-7a9dc0fc674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.251070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686f46d-ab0c-47cf-9659-7a9dc0fc674e admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.251357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686f46d-ab0c-47cf-9659-7a9dc0fc674e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.251632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686f46d-ab0c-47cf-9659-7a9dc0fc674e admin admin] PATH_INFO: `/v3/projects/7a23fa82f0ac49d99b7d29fee241bad4/users/84153330b01b49dfa4c1e0bfcd5ff869/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.268648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686f46d-ab0c-47cf-9659-7a9dc0fc674e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a23fa82f0ac49d99b7d29fee241bad4, user_id=84153330b01b49dfa4c1e0bfcd5ff869, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.271549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686f46d-ab0c-47cf-9659-7a9dc0fc674e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.348832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 981/1947] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:44 2022] PUT /identity/v3/projects/7a23fa82f0ac49d99b7d29fee241bad4/users/84153330b01b49dfa4c1e0bfcd5ff869/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:44.359485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20676a2-892d-492f-95d1-50ec295a4057 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.359903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20676a2-892d-492f-95d1-50ec295a4057 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.360219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20676a2-892d-492f-95d1-50ec295a4057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.389901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d20676a2-892d-492f-95d1-50ec295a4057 None None] MFA Rules not processed for user `84153330b01b49dfa4c1e0bfcd5ff869`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:44.391154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d20676a2-892d-492f-95d1-50ec295a4057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:44.455409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 967/1948] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:53:44.459842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686f46d-ab0c-47cf-9659-7a9dc0fc674e admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.473890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:44.529376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686f46d-ab0c-47cf-9659-7a9dc0fc674e admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.535097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ae87e-dd04-4119-ac9e-b8c5d996ffbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.535847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ae87e-dd04-4119-ac9e-b8c5d996ffbb admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.536125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ae87e-dd04-4119-ac9e-b8c5d996ffbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.536425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ae87e-dd04-4119-ac9e-b8c5d996ffbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.537068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ae87e-dd04-4119-ac9e-b8c5d996ffbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.539983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ae87e-dd04-4119-ac9e-b8c5d996ffbb admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.587230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20676a2-892d-492f-95d1-50ec295a4057 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.588648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d20676a2-892d-492f-95d1-50ec295a4057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:44.603500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20676a2-892d-492f-95d1-50ec295a4057 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.608115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.608751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.609068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.609359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] PATH_INFO: `/v3/users/c07f736d7705478885919532338cc5a9` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.610513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c07f736d7705478885919532338cc5a9)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.621581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.621581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:44.658327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 982/1949] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:44 2022] POST /identity/v3/projects => generated 365 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:44.666749 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ae87e-dd04-4119-ac9e-b8c5d996ffbb admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.731218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ae87e-dd04-4119-ac9e-b8c5d996ffbb admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.736463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b628a-dd71-4395-8521-fddb2e9dd294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.737081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b628a-dd71-4395-8521-fddb2e9dd294 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.737363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b628a-dd71-4395-8521-fddb2e9dd294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.737674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b628a-dd71-4395-8521-fddb2e9dd294 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.738331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b628a-dd71-4395-8521-fddb2e9dd294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.740634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b628a-dd71-4395-8521-fddb2e9dd294 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.871678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 983/1950] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:44 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:53:44.872736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b628a-dd71-4395-8521-fddb2e9dd294 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.876893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c07f736d7705478885919532338cc5a9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:44.883211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b628a-dd71-4395-8521-fddb2e9dd294 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.888158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4dcf99-941d-4ddc-bc12-cabbe9ef856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.888775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dcf99-941d-4ddc-bc12-cabbe9ef856b admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.889105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dcf99-941d-4ddc-bc12-cabbe9ef856b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.889386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4dcf99-941d-4ddc-bc12-cabbe9ef856b admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.890042 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4dcf99-941d-4ddc-bc12-cabbe9ef856b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.891689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4dcf99-941d-4ddc-bc12-cabbe9ef856b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.903585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 984/1951] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:44.908032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4dcf99-941d-4ddc-bc12-cabbe9ef856b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.917793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4dcf99-941d-4ddc-bc12-cabbe9ef856b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.922530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678c138-bc00-4602-8f8e-48b1234f1d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.923233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678c138-bc00-4602-8f8e-48b1234f1d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.923515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678c138-bc00-4602-8f8e-48b1234f1d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.923792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678c138-bc00-4602-8f8e-48b1234f1d57 admin admin] PATH_INFO: `/v3/projects/80d05b42e73441ac95b2e5bfd6722758/users/9a79a04d1ace4d36bc11d663974f6d18/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.939392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678c138-bc00-4602-8f8e-48b1234f1d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d05b42e73441ac95b2e5bfd6722758, user_id=9a79a04d1ace4d36bc11d663974f6d18, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.942151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678c138-bc00-4602-8f8e-48b1234f1d57 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.947927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c07f736d7705478885919532338cc5a9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:44.961580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c07f736d7705478885919532338cc5a9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:44.969671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c07f736d7705478885919532338cc5a9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:44.970640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fb7c0050-cbec-49e1-98a5-80d215b63642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c07f736d7705478885919532338cc5a9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:44.977766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 968/1952] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:44 2022] DELETE /identity/v3/users/c07f736d7705478885919532338cc5a9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:44.982468 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46d15-1d60-49ec-a04b-cbd8e447dfe3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.982855 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46d15-1d60-49ec-a04b-cbd8e447dfe3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.983223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46d15-1d60-49ec-a04b-cbd8e447dfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:45.002816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-1de46d15-1d60-49ec-a04b-cbd8e447dfe3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:45.004082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1de46d15-1d60-49ec-a04b-cbd8e447dfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:45.065893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 969/1953] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:53:45.077205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46d15-1d60-49ec-a04b-cbd8e447dfe3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:45.079448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1de46d15-1d60-49ec-a04b-cbd8e447dfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:45.083493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 985/1954] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:44 2022] PUT /identity/v3/projects/80d05b42e73441ac95b2e5bfd6722758/users/9a79a04d1ace4d36bc11d663974f6d18/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:45.088013 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678c138-bc00-4602-8f8e-48b1234f1d57 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:45.090564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d678c138-bc00-4602-8f8e-48b1234f1d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:45.155628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46d15-1d60-49ec-a04b-cbd8e447dfe3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:45.157273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:45.168845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678c138-bc00-4602-8f8e-48b1234f1d57 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:45.173536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b822977-bc4b-4925-abf6-7f911a5605a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:45.174146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b822977-bc4b-4925-abf6-7f911a5605a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:45.174768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b822977-bc4b-4925-abf6-7f911a5605a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:45.174768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b822977-bc4b-4925-abf6-7f911a5605a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:45.175364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b822977-bc4b-4925-abf6-7f911a5605a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:45.176969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b822977-bc4b-4925-abf6-7f911a5605a2 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:45.192033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 986/1955] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:45.193421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b822977-bc4b-4925-abf6-7f911a5605a2 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:45.199258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:45.204641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b822977-bc4b-4925-abf6-7f911a5605a2 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:45.211603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb923fbc-7a19-42c7-bf9f-d0e2f422409d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:45.212436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb923fbc-7a19-42c7-bf9f-d0e2f422409d service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:45.212660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb923fbc-7a19-42c7-bf9f-d0e2f422409d service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:45.212874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbacd8-4f43-4d86-95ed-f03ebee53a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:45.212988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb923fbc-7a19-42c7-bf9f-d0e2f422409d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:45.213791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbacd8-4f43-4d86-95ed-f03ebee53a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:45.214143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbacd8-4f43-4d86-95ed-f03ebee53a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:45.214533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbacd8-4f43-4d86-95ed-f03ebee53a46 admin admin] PATH_INFO: `/v3/projects/80d05b42e73441ac95b2e5bfd6722758/users/9a79a04d1ace4d36bc11d663974f6d18/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:45.219936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbacd8-4f43-4d86-95ed-f03ebee53a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d05b42e73441ac95b2e5bfd6722758, user_id=9a79a04d1ace4d36bc11d663974f6d18, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:45.223069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbacd8-4f43-4d86-95ed-f03ebee53a46 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:45.224794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb923fbc-7a19-42c7-bf9f-d0e2f422409d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:45.226338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb923fbc-7a19-42c7-bf9f-d0e2f422409d service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:45.240584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 970/1956] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:53:45.245377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 987/1957] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:45 2022] PUT /identity/v3/projects/80d05b42e73441ac95b2e5bfd6722758/users/9a79a04d1ace4d36bc11d663974f6d18/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:45.258085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42034ac-ea58-4416-9016-ca440dd08c1a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:45.258341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42034ac-ea58-4416-9016-ca440dd08c1a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:45.258648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42034ac-ea58-4416-9016-ca440dd08c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:45.293150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-a42034ac-ea58-4416-9016-ca440dd08c1a None None] MFA Rules not processed for user `9a79a04d1ace4d36bc11d663974f6d18`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:45.294411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a42034ac-ea58-4416-9016-ca440dd08c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:45.369431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 971/1958] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:45.408539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbacd8-4f43-4d86-95ed-f03ebee53a46 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:45.410799 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-37dbacd8-4f43-4d86-95ed-f03ebee53a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:45.474054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbacd8-4f43-4d86-95ed-f03ebee53a46 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:45.480735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eb9f8-f202-492b-ab14-76d11b5741ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:45.481477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eb9f8-f202-492b-ab14-76d11b5741ea service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:45.481877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eb9f8-f202-492b-ab14-76d11b5741ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:45.482540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eb9f8-f202-492b-ab14-76d11b5741ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:45.495046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1eb9f8-f202-492b-ab14-76d11b5741ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:45.496952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1eb9f8-f202-492b-ab14-76d11b5741ea service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:45.509721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 988/1959] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:45.720668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42034ac-ea58-4416-9016-ca440dd08c1a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:45.722867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a42034ac-ea58-4416-9016-ca440dd08c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:45.753727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eb9f8-f202-492b-ab14-76d11b5741ea service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:45.764304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eb9f8-f202-492b-ab14-76d11b5741ea service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:45.769326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227a2f3-8005-4cc5-b469-3830bfd46d03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:45.770139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227a2f3-8005-4cc5-b469-3830bfd46d03 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:45.770659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227a2f3-8005-4cc5-b469-3830bfd46d03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:45.771261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227a2f3-8005-4cc5-b469-3830bfd46d03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:45.775275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0227a2f3-8005-4cc5-b469-3830bfd46d03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:45.842162 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42034ac-ea58-4416-9016-ca440dd08c1a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:45.846984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:45.847636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:45.847984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:45.848213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] PATH_INFO: `/v3/projects/73ca51287a13466b93b8fe176a752e75` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:45.849341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ca51287a13466b93b8fe176a752e75)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:45.852147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:45.854343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227a2f3-8005-4cc5-b469-3830bfd46d03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:45.855832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227a2f3-8005-4cc5-b469-3830bfd46d03 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:45.868792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 989/1960] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:45.963789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ca51287a13466b93b8fe176a752e75', 'request_id': 'req-c53df16a-a7d0-4ada-922f-e4030bfce484'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:46.078962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ca51287a13466b93b8fe176a752e75', 'request_id': 'req-c53df16a-a7d0-4ada-922f-e4030bfce484'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:46.086705 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ca51287a13466b93b8fe176a752e75', 'request_id': 'req-c53df16a-a7d0-4ada-922f-e4030bfce484'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:46.093517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:46.098581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] The token cache is being invalidate because project 73ca51287a13466b93b8fe176a752e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:46.098581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:46.099140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c53df16a-a7d0-4ada-922f-e4030bfce484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:46.102532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 972/1961] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:45 2022] DELETE /identity/v3/projects/73ca51287a13466b93b8fe176a752e75 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:46.106958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdda6c-4d4f-4b70-a50c-a2c90143d150 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:46.107273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdda6c-4d4f-4b70-a50c-a2c90143d150 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:46.107558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdda6c-4d4f-4b70-a50c-a2c90143d150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:46.127730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-20cdda6c-4d4f-4b70-a50c-a2c90143d150 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:46.129018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-20cdda6c-4d4f-4b70-a50c-a2c90143d150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:46.199303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 990/1962] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:53:46.208808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6bba7-445e-4ce2-a2d6-7d41fd3c8a77 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:46.209151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6bba7-445e-4ce2-a2d6-7d41fd3c8a77 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:46.209438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6bba7-445e-4ce2-a2d6-7d41fd3c8a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:46.229767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-68b6bba7-445e-4ce2-a2d6-7d41fd3c8a77 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:46.231055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-68b6bba7-445e-4ce2-a2d6-7d41fd3c8a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:46.239959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 973/1963] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:46.244060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdda6c-4d4f-4b70-a50c-a2c90143d150 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:46.254370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdda6c-4d4f-4b70-a50c-a2c90143d150 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:46.260059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bfacb-3da5-4d5c-8316-60b53920a695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:46.260469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bfacb-3da5-4d5c-8316-60b53920a695 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:46.260759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bfacb-3da5-4d5c-8316-60b53920a695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:46.261035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bfacb-3da5-4d5c-8316-60b53920a695 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:46.261690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bfacb-3da5-4d5c-8316-60b53920a695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:53:46.261996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bfacb-3da5-4d5c-8316-60b53920a695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:46.263656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bfacb-3da5-4d5c-8316-60b53920a695 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:46.278216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 991/1964] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:53:46.290344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6bba7-445e-4ce2-a2d6-7d41fd3c8a77 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:46.304395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6bba7-445e-4ce2-a2d6-7d41fd3c8a77 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:46.313056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1660ed8-b58b-46bb-9b5d-831547c694c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:46.316181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1660ed8-b58b-46bb-9b5d-831547c694c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:46.316181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1660ed8-b58b-46bb-9b5d-831547c694c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:46.316181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1660ed8-b58b-46bb-9b5d-831547c694c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:46.316181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1660ed8-b58b-46bb-9b5d-831547c694c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:46.318402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1660ed8-b58b-46bb-9b5d-831547c694c6 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:46.414219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 974/1965] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:46 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:46.417061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bfacb-3da5-4d5c-8316-60b53920a695 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:46.423638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:46.429671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bfacb-3da5-4d5c-8316-60b53920a695 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:46.501123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5eb4b-01ed-43f3-ab3a-8394c83f0301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:46.501777 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5eb4b-01ed-43f3-ab3a-8394c83f0301 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:46.502058 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5eb4b-01ed-43f3-ab3a-8394c83f0301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:46.502334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5eb4b-01ed-43f3-ab3a-8394c83f0301 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:46.503046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5eb4b-01ed-43f3-ab3a-8394c83f0301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:46.505037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5eb4b-01ed-43f3-ab3a-8394c83f0301 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:46.644028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 992/1966] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:46 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:46.645215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1660ed8-b58b-46bb-9b5d-831547c694c6 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:46.655934 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1660ed8-b58b-46bb-9b5d-831547c694c6 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:46.660492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8083f7-3d26-41ac-b043-bf534a10a599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:46.661087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8083f7-3d26-41ac-b043-bf534a10a599 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:46.661368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8083f7-3d26-41ac-b043-bf534a10a599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:46.661643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8083f7-3d26-41ac-b043-bf534a10a599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:46.662287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8083f7-3d26-41ac-b043-bf534a10a599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:46.664212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8083f7-3d26-41ac-b043-bf534a10a599 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:46.674009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 975/1967] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:53:46.678110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5eb4b-01ed-43f3-ab3a-8394c83f0301 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:46.688760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5eb4b-01ed-43f3-ab3a-8394c83f0301 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:46.693393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3056a60-2b43-4fea-85fe-fd8941b2f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:46.694003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056a60-2b43-4fea-85fe-fd8941b2f2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:46.694286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056a60-2b43-4fea-85fe-fd8941b2f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:46.695178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056a60-2b43-4fea-85fe-fd8941b2f2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:46.698391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3056a60-2b43-4fea-85fe-fd8941b2f2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:46.698391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3056a60-2b43-4fea-85fe-fd8941b2f2b9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:46.716721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 993/1968] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:46.718206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8083f7-3d26-41ac-b043-bf534a10a599 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:46.729395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8083f7-3d26-41ac-b043-bf534a10a599 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:46.734315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4425e-284b-40bc-99e1-f5c1ffbc1c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:46.734974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4425e-284b-40bc-99e1-f5c1ffbc1c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:46.735282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4425e-284b-40bc-99e1-f5c1ffbc1c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:46.735567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4425e-284b-40bc-99e1-f5c1ffbc1c7d admin admin] PATH_INFO: `/v3/projects/82af5eacc6004249910b0e3cbd8b7f34/users/a561c97996cc4677823df848773232e6/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:46.752199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4425e-284b-40bc-99e1-f5c1ffbc1c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82af5eacc6004249910b0e3cbd8b7f34, user_id=a561c97996cc4677823df848773232e6, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:46.755191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4425e-284b-40bc-99e1-f5c1ffbc1c7d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:46.832433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 976/1969] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:46 2022] PUT /identity/v3/projects/82af5eacc6004249910b0e3cbd8b7f34/users/a561c97996cc4677823df848773232e6/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:46.843607 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3099735-9e78-4fd0-88ef-7c45a4566ac8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:46.844079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3099735-9e78-4fd0-88ef-7c45a4566ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:46.844194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3099735-9e78-4fd0-88ef-7c45a4566ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:46.873505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b3099735-9e78-4fd0-88ef-7c45a4566ac8 None None] MFA Rules not processed for user `a561c97996cc4677823df848773232e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:46.874821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b3099735-9e78-4fd0-88ef-7c45a4566ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:46.937775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 994/1970] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:46.948813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4425e-284b-40bc-99e1-f5c1ffbc1c7d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:46.950540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-40d4425e-284b-40bc-99e1-f5c1ffbc1c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:47.013390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4425e-284b-40bc-99e1-f5c1ffbc1c7d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:47.020781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aeb20e-42c2-423e-8ae6-39c0277799f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:47.021409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeb20e-42c2-423e-8ae6-39c0277799f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:47.022018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeb20e-42c2-423e-8ae6-39c0277799f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:47.022018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeb20e-42c2-423e-8ae6-39c0277799f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:47.031763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeb20e-42c2-423e-8ae6-39c0277799f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:47.033255 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeb20e-42c2-423e-8ae6-39c0277799f3 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:47.043218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 977/1971] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:47.789698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3099735-9e78-4fd0-88ef-7c45a4566ac8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:47.791213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b3099735-9e78-4fd0-88ef-7c45a4566ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:47.861602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3099735-9e78-4fd0-88ef-7c45a4566ac8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:47.866900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:47.867526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:47.867801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:47.868074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] PATH_INFO: `/v3/users/3ff6642706b74460badcad0eed3c7de6` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:47.869204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff6642706b74460badcad0eed3c7de6)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:47.871168 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:47.885492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:48.054923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff6642706b74460badcad0eed3c7de6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:48.055659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff6642706b74460badcad0eed3c7de6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:48.169885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff6642706b74460badcad0eed3c7de6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:48.177584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff6642706b74460badcad0eed3c7de6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:48.191359 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff6642706b74460badcad0eed3c7de6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:48.197883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 995/1972] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:47 2022] DELETE /identity/v3/users/3ff6642706b74460badcad0eed3c7de6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:48.202834 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bfd3-5e09-45fe-9b09-e6b0ade33f72 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:48.203663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bfd3-5e09-45fe-9b09-e6b0ade33f72 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:48.203946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bfd3-5e09-45fe-9b09-e6b0ade33f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:48.224444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d5f4bfd3-5e09-45fe-9b09-e6b0ade33f72 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:48.233587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d5f4bfd3-5e09-45fe-9b09-e6b0ade33f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:48.298458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 978/1973] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:48.308847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:48.310337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:48.371120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472d00a-e53a-437a-a8fe-e1343f56d552 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:48.374943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a931d9-6bad-4560-9eb6-a178654b05ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:48.375550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a931d9-6bad-4560-9eb6-a178654b05ea service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:48.375827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a931d9-6bad-4560-9eb6-a178654b05ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:48.376101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a931d9-6bad-4560-9eb6-a178654b05ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:48.385703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a931d9-6bad-4560-9eb6-a178654b05ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:48.387192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a931d9-6bad-4560-9eb6-a178654b05ea service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:48.396650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 996/1974] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:48.847050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98471756-34d7-400d-85bc-b7815d356ff9 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:48.847338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98471756-34d7-400d-85bc-b7815d356ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:48.847621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98471756-34d7-400d-85bc-b7815d356ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:48.868039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-98471756-34d7-400d-85bc-b7815d356ff9 None None] MFA Rules not processed for user `a561c97996cc4677823df848773232e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:48.869035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-98471756-34d7-400d-85bc-b7815d356ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:48.886161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a931d9-6bad-4560-9eb6-a178654b05ea service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:48.889243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-39a931d9-6bad-4560-9eb6-a178654b05ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:48.906490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a931d9-6bad-4560-9eb6-a178654b05ea service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:48.908799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:48.909008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:48.909649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:48.909935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:48.910243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] PATH_INFO: `/v3/projects/dc09c90dfe2a43d7a32329ad903bbf26` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:48.911427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc09c90dfe2a43d7a32329ad903bbf26)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:48.913590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:48.932101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 979/1975] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:49.027352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc09c90dfe2a43d7a32329ad903bbf26', 'request_id': 'req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:49.033745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc09c90dfe2a43d7a32329ad903bbf26', 'request_id': 'req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:49.106383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc09c90dfe2a43d7a32329ad903bbf26', 'request_id': 'req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:49.119715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] The token cache is being invalidate because project dc09c90dfe2a43d7a32329ad903bbf26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:49.120008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:49.120673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3aa4c82f-d98b-49ed-bc46-40b19d5186b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:49.122239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 997/1976] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:48 2022] DELETE /identity/v3/projects/dc09c90dfe2a43d7a32329ad903bbf26 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:53:49.127291 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dce9d-b3d5-481d-8ed6-fb7fbbcddcc9 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:49.127767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dce9d-b3d5-481d-8ed6-fb7fbbcddcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:49.128043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dce9d-b3d5-481d-8ed6-fb7fbbcddcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:49.148147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-db1dce9d-b3d5-481d-8ed6-fb7fbbcddcc9 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:49.149563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-db1dce9d-b3d5-481d-8ed6-fb7fbbcddcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:49.227615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 980/1977] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:49.235029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e9418-c37b-4aea-be43-9dba0d5a8592 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:49.235324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e9418-c37b-4aea-be43-9dba0d5a8592 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:49.235613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e9418-c37b-4aea-be43-9dba0d5a8592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:49.257227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-082e9418-c37b-4aea-be43-9dba0d5a8592 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:49.258460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-082e9418-c37b-4aea-be43-9dba0d5a8592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:49.266646 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 998/1978] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:53:49.274195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1dce9d-b3d5-481d-8ed6-fb7fbbcddcc9 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:49.285843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1dce9d-b3d5-481d-8ed6-fb7fbbcddcc9 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:49.291397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1ffe6-de5a-4f24-95ac-8cb8f3822025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:49.292018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1ffe6-de5a-4f24-95ac-8cb8f3822025 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:49.292298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1ffe6-de5a-4f24-95ac-8cb8f3822025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:49.292569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1ffe6-de5a-4f24-95ac-8cb8f3822025 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:49.293221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1ffe6-de5a-4f24-95ac-8cb8f3822025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:53:49.293518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1ffe6-de5a-4f24-95ac-8cb8f3822025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:49.295194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1ffe6-de5a-4f24-95ac-8cb8f3822025 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:49.309562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 981/1979] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:53:49.313848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e9418-c37b-4aea-be43-9dba0d5a8592 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:49.324910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e9418-c37b-4aea-be43-9dba0d5a8592 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:49.329735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c76c0-637f-4bc4-a924-3057333903d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:49.330371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c76c0-637f-4bc4-a924-3057333903d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:49.330689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c76c0-637f-4bc4-a924-3057333903d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:49.330980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c76c0-637f-4bc4-a924-3057333903d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:49.331622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c76c0-637f-4bc4-a924-3057333903d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:49.333543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c76c0-637f-4bc4-a924-3057333903d4 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:49.343878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:49.450305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 999/1980] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:49 2022] POST /identity/v3/projects => generated 397 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:49.451434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1ffe6-de5a-4f24-95ac-8cb8f3822025 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:49.463422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1ffe6-de5a-4f24-95ac-8cb8f3822025 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:49.532180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8756782-8daa-4514-b8c3-c60e765d3a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:49.532802 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8756782-8daa-4514-b8c3-c60e765d3a34 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:49.533087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8756782-8daa-4514-b8c3-c60e765d3a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:49.533366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8756782-8daa-4514-b8c3-c60e765d3a34 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:49.534198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8756782-8daa-4514-b8c3-c60e765d3a34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:49.536038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8756782-8daa-4514-b8c3-c60e765d3a34 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:49.677402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 982/1981] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:49 2022] POST /identity/v3/users => generated 299 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:49.678772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c76c0-637f-4bc4-a924-3057333903d4 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:49.691033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c76c0-637f-4bc4-a924-3057333903d4 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:49.697669 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b0e58-3da0-46aa-835c-3db24e18e61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:49.698498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b0e58-3da0-46aa-835c-3db24e18e61a admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:49.698769 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b0e58-3da0-46aa-835c-3db24e18e61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:49.699383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b0e58-3da0-46aa-835c-3db24e18e61a admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:49.699959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b0e58-3da0-46aa-835c-3db24e18e61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:49.701635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b0e58-3da0-46aa-835c-3db24e18e61a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:49.714579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1000/1982] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:49.720384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8756782-8daa-4514-b8c3-c60e765d3a34 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:49.731751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8756782-8daa-4514-b8c3-c60e765d3a34 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:49.737487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a0ca3-3b4c-4c8f-b861-aa665924e6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:49.738160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a0ca3-3b4c-4c8f-b861-aa665924e6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:49.738448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a0ca3-3b4c-4c8f-b861-aa665924e6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:49.738762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a0ca3-3b4c-4c8f-b861-aa665924e6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:49.739419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a0ca3-3b4c-4c8f-b861-aa665924e6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:49.741345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a0ca3-3b4c-4c8f-b861-aa665924e6e5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:49.755926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 983/1983] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:49.761647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b0e58-3da0-46aa-835c-3db24e18e61a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:49.771855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b0e58-3da0-46aa-835c-3db24e18e61a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:49.778708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09feafbf-3f21-45f0-885f-90979b594c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:49.779367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09feafbf-3f21-45f0-885f-90979b594c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:49.779856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09feafbf-3f21-45f0-885f-90979b594c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:49.780129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09feafbf-3f21-45f0-885f-90979b594c2a admin admin] PATH_INFO: `/v3/projects/4bceff84a32f4e90a6407d5bcdda9946/users/f88457b7f7f5498785e8cddc498c0558/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:49.799284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09feafbf-3f21-45f0-885f-90979b594c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bceff84a32f4e90a6407d5bcdda9946, user_id=f88457b7f7f5498785e8cddc498c0558, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:49.802255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09feafbf-3f21-45f0-885f-90979b594c2a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:49.886770 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1001/1984] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:49 2022] PUT /identity/v3/projects/4bceff84a32f4e90a6407d5bcdda9946/users/f88457b7f7f5498785e8cddc498c0558/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:49.898150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c7b8-2af4-49d4-958e-ec38b9609284 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:49.899178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c7b8-2af4-49d4-958e-ec38b9609284 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:49.899667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c7b8-2af4-49d4-958e-ec38b9609284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:49.928283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c363c7b8-2af4-49d4-958e-ec38b9609284 None None] MFA Rules not processed for user `f88457b7f7f5498785e8cddc498c0558`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:49.930180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c363c7b8-2af4-49d4-958e-ec38b9609284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:49.972647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:50.005145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 984/1985] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:50.018388 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09feafbf-3f21-45f0-885f-90979b594c2a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:50.020523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-09feafbf-3f21-45f0-885f-90979b594c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:50.081115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09feafbf-3f21-45f0-885f-90979b594c2a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:50.088170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef5fb5-a880-412f-a59a-4634f5643750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:50.088170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5fb5-a880-412f-a59a-4634f5643750 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:50.088170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5fb5-a880-412f-a59a-4634f5643750 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:50.088170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5fb5-a880-412f-a59a-4634f5643750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:50.095868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef5fb5-a880-412f-a59a-4634f5643750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:50.097315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef5fb5-a880-412f-a59a-4634f5643750 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:50.106649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1002/1986] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:50 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:54.722863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363c7b8-2af4-49d4-958e-ec38b9609284 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:54.725004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c363c7b8-2af4-49d4-958e-ec38b9609284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:54.800371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363c7b8-2af4-49d4-958e-ec38b9609284 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:54.806155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:54.807193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:54.807511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:54.807982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] PATH_INFO: `/v3/users/35ce0f9dad99470cb666e685021216b2` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:54.809519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35ce0f9dad99470cb666e685021216b2)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:54.810989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:55.034636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35ce0f9dad99470cb666e685021216b2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:55.139857 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35ce0f9dad99470cb666e685021216b2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:55.159952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35ce0f9dad99470cb666e685021216b2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:55.167295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35ce0f9dad99470cb666e685021216b2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:55.168049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35ce0f9dad99470cb666e685021216b2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:55.175977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 985/1987] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:54 2022] DELETE /identity/v3/users/35ce0f9dad99470cb666e685021216b2 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:55.178862 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a58109-01b3-4b2b-b991-bdbe71b76019 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:55.179342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a58109-01b3-4b2b-b991-bdbe71b76019 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:55.179342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a58109-01b3-4b2b-b991-bdbe71b76019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:55.199802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d5a58109-01b3-4b2b-b991-bdbe71b76019 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:55.200952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d5a58109-01b3-4b2b-b991-bdbe71b76019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:55.276177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1003/1988] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:55.286159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:55.287675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:55.354895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c5c6d-baad-4920-9dd7-66015a9b7ec0 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:55.361033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2750b-f78c-4283-a7c0-6c3ea93c9344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:55.361691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2750b-f78c-4283-a7c0-6c3ea93c9344 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:55.361975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2750b-f78c-4283-a7c0-6c3ea93c9344 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:55.362253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2750b-f78c-4283-a7c0-6c3ea93c9344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:55.372596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2750b-f78c-4283-a7c0-6c3ea93c9344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:55.374050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2750b-f78c-4283-a7c0-6c3ea93c9344 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:55.379572 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:55.384381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 986/1989] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:55.858377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a58109-01b3-4b2b-b991-bdbe71b76019 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:55.859937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d5a58109-01b3-4b2b-b991-bdbe71b76019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:55.875323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a58109-01b3-4b2b-b991-bdbe71b76019 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:55.883326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:55.884431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:55.888813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:55.888813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] PATH_INFO: `/v3/projects/82086ddb29eb4893b771c71b725d930a` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:55.888813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=82086ddb29eb4893b771c71b725d930a)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:55.890894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:55.982274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82086ddb29eb4893b771c71b725d930a', 'request_id': 'req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:55.984276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:55.988089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82086ddb29eb4893b771c71b725d930a', 'request_id': 'req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:56.077859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82086ddb29eb4893b771c71b725d930a', 'request_id': 'req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:56.090500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] The token cache is being invalidate because project 82086ddb29eb4893b771c71b725d930a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:56.090821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:56.091481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-398e06b0-2eff-4d27-9c5b-2a2fe6f40a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:56.092996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1004/1990] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:55 2022] DELETE /identity/v3/projects/82086ddb29eb4893b771c71b725d930a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:56.098188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a34aa-6674-4ed4-a822-0997f46f6991 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:56.098711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a34aa-6674-4ed4-a822-0997f46f6991 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:56.098802 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a34aa-6674-4ed4-a822-0997f46f6991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:56.119411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-865a34aa-6674-4ed4-a822-0997f46f6991 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:56.120625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-865a34aa-6674-4ed4-a822-0997f46f6991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:56.184246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 987/1991] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:56.193160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4beea4c-2105-49ff-82dc-cf20f381140f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:56.193673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4beea4c-2105-49ff-82dc-cf20f381140f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:56.193783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4beea4c-2105-49ff-82dc-cf20f381140f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:56.214401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e4beea4c-2105-49ff-82dc-cf20f381140f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:56.215694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e4beea4c-2105-49ff-82dc-cf20f381140f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:56.223634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1005/1992] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:56.228664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a34aa-6674-4ed4-a822-0997f46f6991 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:56.238610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a34aa-6674-4ed4-a822-0997f46f6991 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:56.243559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710eca6d-17f7-4fbb-a02d-72c65e901e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:56.244171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eca6d-17f7-4fbb-a02d-72c65e901e96 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:56.244455 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eca6d-17f7-4fbb-a02d-72c65e901e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:56.244734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eca6d-17f7-4fbb-a02d-72c65e901e96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:56.245388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710eca6d-17f7-4fbb-a02d-72c65e901e96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:53:56.245698 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710eca6d-17f7-4fbb-a02d-72c65e901e96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:56.247386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710eca6d-17f7-4fbb-a02d-72c65e901e96 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:56.261599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 988/1993] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:53:56.263624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4beea4c-2105-49ff-82dc-cf20f381140f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:56.273146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4beea4c-2105-49ff-82dc-cf20f381140f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:56.279224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bda54-d442-46b0-85ea-ac49e7808180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:56.284610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bda54-d442-46b0-85ea-ac49e7808180 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:56.284610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bda54-d442-46b0-85ea-ac49e7808180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:56.284610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bda54-d442-46b0-85ea-ac49e7808180 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:56.284610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bda54-d442-46b0-85ea-ac49e7808180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:56.284610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bda54-d442-46b0-85ea-ac49e7808180 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:56.373042 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1006/1994] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:56 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:56.378852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710eca6d-17f7-4fbb-a02d-72c65e901e96 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:56.392342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710eca6d-17f7-4fbb-a02d-72c65e901e96 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:56.469269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fc304-f18a-47d2-b249-de651c1c2376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:56.471831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fc304-f18a-47d2-b249-de651c1c2376 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:56.471831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fc304-f18a-47d2-b249-de651c1c2376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:56.471831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fc304-f18a-47d2-b249-de651c1c2376 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:56.471831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fc304-f18a-47d2-b249-de651c1c2376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:56.479140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fc304-f18a-47d2-b249-de651c1c2376 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:56.575843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 989/1995] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:56 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:53:56.580304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bda54-d442-46b0-85ea-ac49e7808180 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:56.593465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bda54-d442-46b0-85ea-ac49e7808180 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:56.598624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4a598-2719-401f-a45c-75e7a858ee73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:56.599426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4a598-2719-401f-a45c-75e7a858ee73 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:56.599869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4a598-2719-401f-a45c-75e7a858ee73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:56.600211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4a598-2719-401f-a45c-75e7a858ee73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:56.600944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4a598-2719-401f-a45c-75e7a858ee73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:56.603009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4a598-2719-401f-a45c-75e7a858ee73 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:56.613972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1007/1996] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:56.618283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fc304-f18a-47d2-b249-de651c1c2376 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:56.630649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fc304-f18a-47d2-b249-de651c1c2376 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:56.636546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7800be5-7f0c-439a-b052-ca52268360a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:56.637243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7800be5-7f0c-439a-b052-ca52268360a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:56.637424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7800be5-7f0c-439a-b052-ca52268360a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:56.637710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7800be5-7f0c-439a-b052-ca52268360a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:56.638373 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7800be5-7f0c-439a-b052-ca52268360a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:56.640046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7800be5-7f0c-439a-b052-ca52268360a3 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:56.655732 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 990/1997] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:53:56.660061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4a598-2719-401f-a45c-75e7a858ee73 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:56.675636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4a598-2719-401f-a45c-75e7a858ee73 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:56.684643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17709975-975f-4149-b0de-6e780bfb5e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:56.685364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17709975-975f-4149-b0de-6e780bfb5e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:56.685653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17709975-975f-4149-b0de-6e780bfb5e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:56.685961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17709975-975f-4149-b0de-6e780bfb5e4d admin admin] PATH_INFO: `/v3/projects/03ecd1067eb14f1e85a681090bc8fd20/users/c5551ecda2f94eb79b0ffbb07751849f/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:56.702534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17709975-975f-4149-b0de-6e780bfb5e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ecd1067eb14f1e85a681090bc8fd20, user_id=c5551ecda2f94eb79b0ffbb07751849f, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:56.705335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17709975-975f-4149-b0de-6e780bfb5e4d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:56.823908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1008/1998] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:56 2022] PUT /identity/v3/projects/03ecd1067eb14f1e85a681090bc8fd20/users/c5551ecda2f94eb79b0ffbb07751849f/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:56.834625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c76917-05b5-4331-a3c4-26f3cd569fb7 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:56.834920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c76917-05b5-4331-a3c4-26f3cd569fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:56.835236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c76917-05b5-4331-a3c4-26f3cd569fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:56.867233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-48c76917-05b5-4331-a3c4-26f3cd569fb7 None None] MFA Rules not processed for user `c5551ecda2f94eb79b0ffbb07751849f`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:56.869620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-48c76917-05b5-4331-a3c4-26f3cd569fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:56.935022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:56.947383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 991/1999] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:56.952337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17709975-975f-4149-b0de-6e780bfb5e4d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:57.018284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17709975-975f-4149-b0de-6e780bfb5e4d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:57.023423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faefb97-fee0-4042-8245-7137a8dfe54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:57.024077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faefb97-fee0-4042-8245-7137a8dfe54b admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:57.024361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faefb97-fee0-4042-8245-7137a8dfe54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:57.024797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faefb97-fee0-4042-8245-7137a8dfe54b admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:57.025474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faefb97-fee0-4042-8245-7137a8dfe54b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:57.027431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faefb97-fee0-4042-8245-7137a8dfe54b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:57.112984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1009/2000] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:56 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:57.117287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c76917-05b5-4331-a3c4-26f3cd569fb7 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:57.129208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c76917-05b5-4331-a3c4-26f3cd569fb7 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:57.189297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d097217-e363-4fdc-8a22-8711c13adfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:57.189920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d097217-e363-4fdc-8a22-8711c13adfcd admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:57.190207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d097217-e363-4fdc-8a22-8711c13adfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:57.190493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d097217-e363-4fdc-8a22-8711c13adfcd admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:57.191219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d097217-e363-4fdc-8a22-8711c13adfcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:57.193158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d097217-e363-4fdc-8a22-8711c13adfcd admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:57.313146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 992/2001] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:57 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:57.317050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faefb97-fee0-4042-8245-7137a8dfe54b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:57.328221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faefb97-fee0-4042-8245-7137a8dfe54b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:57.333093 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae41bb-7107-4aa1-9bcf-9a8bcb9ff9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:57.333723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae41bb-7107-4aa1-9bcf-9a8bcb9ff9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:57.334005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae41bb-7107-4aa1-9bcf-9a8bcb9ff9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:57.334284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae41bb-7107-4aa1-9bcf-9a8bcb9ff9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:57.334974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae41bb-7107-4aa1-9bcf-9a8bcb9ff9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:57.336850 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae41bb-7107-4aa1-9bcf-9a8bcb9ff9f9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:57.351535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1010/2002] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:57.356223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d097217-e363-4fdc-8a22-8711c13adfcd admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:57.373248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d097217-e363-4fdc-8a22-8711c13adfcd admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:57.377990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed3bbb-75cd-485d-a4ec-01e71c45b863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:57.378690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed3bbb-75cd-485d-a4ec-01e71c45b863 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:57.378973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed3bbb-75cd-485d-a4ec-01e71c45b863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:57.379248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed3bbb-75cd-485d-a4ec-01e71c45b863 admin admin] PATH_INFO: `/v3/projects/671dd7be873547c888bb7f7a341c93c4/users/de80dbfffc324873a09da0104c4a992f/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:57.398124 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed3bbb-75cd-485d-a4ec-01e71c45b863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=671dd7be873547c888bb7f7a341c93c4, user_id=de80dbfffc324873a09da0104c4a992f, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:57.401190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed3bbb-75cd-485d-a4ec-01e71c45b863 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:57.526841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 993/2003] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:57 2022] PUT /identity/v3/projects/671dd7be873547c888bb7f7a341c93c4/users/de80dbfffc324873a09da0104c4a992f/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:57.530909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae41bb-7107-4aa1-9bcf-9a8bcb9ff9f9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:57.542218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae41bb-7107-4aa1-9bcf-9a8bcb9ff9f9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:57.598408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:57.609465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205f2ee-cf06-4309-b447-09ca7a9f2324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:57.612891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205f2ee-cf06-4309-b447-09ca7a9f2324 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:57.612891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205f2ee-cf06-4309-b447-09ca7a9f2324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:57.612891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205f2ee-cf06-4309-b447-09ca7a9f2324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:57.612891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205f2ee-cf06-4309-b447-09ca7a9f2324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:57.613216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205f2ee-cf06-4309-b447-09ca7a9f2324 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:57.623987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1011/2004] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:53:57.630092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed3bbb-75cd-485d-a4ec-01e71c45b863 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:57.642626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed3bbb-75cd-485d-a4ec-01e71c45b863 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:57.649944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76446592-fb2c-4785-ae39-a7cd33932f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:57.650691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76446592-fb2c-4785-ae39-a7cd33932f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:57.650985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76446592-fb2c-4785-ae39-a7cd33932f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:57.652014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76446592-fb2c-4785-ae39-a7cd33932f8d admin admin] PATH_INFO: `/v3/projects/671dd7be873547c888bb7f7a341c93c4/users/de80dbfffc324873a09da0104c4a992f/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:57.653523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76446592-fb2c-4785-ae39-a7cd33932f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=671dd7be873547c888bb7f7a341c93c4, user_id=de80dbfffc324873a09da0104c4a992f, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:57.656561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76446592-fb2c-4785-ae39-a7cd33932f8d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:57.676918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 994/2005] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:57 2022] PUT /identity/v3/projects/671dd7be873547c888bb7f7a341c93c4/users/de80dbfffc324873a09da0104c4a992f/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:57.687633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a562a-934f-4de3-aa9d-886769404d0d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:57.688007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a562a-934f-4de3-aa9d-886769404d0d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:57.688379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a562a-934f-4de3-aa9d-886769404d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:57.723612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-de5a562a-934f-4de3-aa9d-886769404d0d None None] MFA Rules not processed for user `de80dbfffc324873a09da0104c4a992f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:57.726060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-de5a562a-934f-4de3-aa9d-886769404d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:57.823609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1012/2006] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:53:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:57.834924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76446592-fb2c-4785-ae39-a7cd33932f8d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:57.837191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-76446592-fb2c-4785-ae39-a7cd33932f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:57.903423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76446592-fb2c-4785-ae39-a7cd33932f8d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:57.907568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19165dc-5637-4b23-b12c-f801cbb8457e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:57.908193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19165dc-5637-4b23-b12c-f801cbb8457e service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:57.908616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19165dc-5637-4b23-b12c-f801cbb8457e service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:57.908777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19165dc-5637-4b23-b12c-f801cbb8457e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:57.976279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19165dc-5637-4b23-b12c-f801cbb8457e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:57.978427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19165dc-5637-4b23-b12c-f801cbb8457e service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:57.988547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 995/2007] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:01.268332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a562a-934f-4de3-aa9d-886769404d0d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.350270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a562a-934f-4de3-aa9d-886769404d0d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.356098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.356753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.357031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.357308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] PATH_INFO: `/v3/users/c5551ecda2f94eb79b0ffbb07751849f` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.358479 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5551ecda2f94eb79b0ffbb07751849f)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.360484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.456809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19165dc-5637-4b23-b12c-f801cbb8457e service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.458226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f19165dc-5637-4b23-b12c-f801cbb8457e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:01.467837 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:01.473813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19165dc-5637-4b23-b12c-f801cbb8457e service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.479314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.479963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.480245 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.480520 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] PATH_INFO: `/v3/users/f88457b7f7f5498785e8cddc498c0558` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.481663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f88457b7f7f5498785e8cddc498c0558)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.483666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.874198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5551ecda2f94eb79b0ffbb07751849f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:01.874883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5551ecda2f94eb79b0ffbb07751849f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:01.909927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f88457b7f7f5498785e8cddc498c0558'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:01.968409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5551ecda2f94eb79b0ffbb07751849f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:01.976787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5551ecda2f94eb79b0ffbb07751849f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:01.985265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:01.990402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-724db7d1-e4af-4dd3-8fe1-20bc29ddc5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5551ecda2f94eb79b0ffbb07751849f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:01.998925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1013/2008] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:01 2022] DELETE /identity/v3/users/c5551ecda2f94eb79b0ffbb07751849f => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:02.003423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c0bb0-ce1c-4914-b056-bd4655f7d8f8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:02.003716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c0bb0-ce1c-4914-b056-bd4655f7d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:02.004014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c0bb0-ce1c-4914-b056-bd4655f7d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:02.025676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-837c0bb0-ce1c-4914-b056-bd4655f7d8f8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:02.026957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-837c0bb0-ce1c-4914-b056-bd4655f7d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.031692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f88457b7f7f5498785e8cddc498c0558'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:02.048629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f88457b7f7f5498785e8cddc498c0558'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:02.058304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f88457b7f7f5498785e8cddc498c0558'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:02.058814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3a426320-73df-4109-b4cf-b9760fa69f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f88457b7f7f5498785e8cddc498c0558'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:02.065377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 996/2009] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:01 2022] DELETE /identity/v3/users/f88457b7f7f5498785e8cddc498c0558 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:54:02.069651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e46a-0a84-49e4-8f81-c46f3d7d2e35 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:02.070123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e46a-0a84-49e4-8f81-c46f3d7d2e35 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:02.070566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e46a-0a84-49e4-8f81-c46f3d7d2e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:02.099413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-ed81e46a-0a84-49e4-8f81-c46f3d7d2e35 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:02.099413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ed81e46a-0a84-49e4-8f81-c46f3d7d2e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.119559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1014/2010] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:54:02.135531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c0bb0-ce1c-4914-b056-bd4655f7d8f8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:02.138668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-837c0bb0-ce1c-4914-b056-bd4655f7d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.197044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 997/2011] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:54:02.206349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c0bb0-ce1c-4914-b056-bd4655f7d8f8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:02.214126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e46a-0a84-49e4-8f81-c46f3d7d2e35 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:02.214126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ed81e46a-0a84-49e4-8f81-c46f3d7d2e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.279928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edad75-e1d6-4e57-9f75-df8b0c963ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:02.279928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edad75-e1d6-4e57-9f75-df8b0c963ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:02.279928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edad75-e1d6-4e57-9f75-df8b0c963ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:02.279928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edad75-e1d6-4e57-9f75-df8b0c963ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:02.284824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e46a-0a84-49e4-8f81-c46f3d7d2e35 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:02.294093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45598f-efe8-4cd7-b6a0-0bbd550e5330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:02.294093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45598f-efe8-4cd7-b6a0-0bbd550e5330 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:02.294093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45598f-efe8-4cd7-b6a0-0bbd550e5330 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:02.294485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45598f-efe8-4cd7-b6a0-0bbd550e5330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:02.298282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edad75-e1d6-4e57-9f75-df8b0c963ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:02.300107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edad75-e1d6-4e57-9f75-df8b0c963ea2 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:02.305158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45598f-efe8-4cd7-b6a0-0bbd550e5330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:02.306684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45598f-efe8-4cd7-b6a0-0bbd550e5330 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:02.314735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1015/2012] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:02.318562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 998/2013] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:02.498289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edad75-e1d6-4e57-9f75-df8b0c963ea2 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:02.499786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-05edad75-e1d6-4e57-9f75-df8b0c963ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.513532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edad75-e1d6-4e57-9f75-df8b0c963ea2 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:02.518913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:02.519589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:02.522697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:02.522697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] PATH_INFO: `/v3/users/84153330b01b49dfa4c1e0bfcd5ff869` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:02.522697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84153330b01b49dfa4c1e0bfcd5ff869)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:02.524176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:02.784617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45598f-efe8-4cd7-b6a0-0bbd550e5330 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:02.786598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84153330b01b49dfa4c1e0bfcd5ff869'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:02.787911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84153330b01b49dfa4c1e0bfcd5ff869'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:02.789700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ed45598f-efe8-4cd7-b6a0-0bbd550e5330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.803925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45598f-efe8-4cd7-b6a0-0bbd550e5330 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:02.809265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:02.809910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:02.810192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:02.810465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] PATH_INFO: `/v3/projects/4bceff84a32f4e90a6407d5bcdda9946` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:02.811875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bceff84a32f4e90a6407d5bcdda9946)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:02.814986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:02.859593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84153330b01b49dfa4c1e0bfcd5ff869'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:02.868045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84153330b01b49dfa4c1e0bfcd5ff869'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:02.880260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-47a21250-dfee-493e-b713-55c5a1187fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84153330b01b49dfa4c1e0bfcd5ff869'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:02.887527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1016/2014] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:02 2022] DELETE /identity/v3/users/84153330b01b49dfa4c1e0bfcd5ff869 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:02.891817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cffce-9f1a-4991-b088-e50a770eac63 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:02.892143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cffce-9f1a-4991-b088-e50a770eac63 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:02.892523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cffce-9f1a-4991-b088-e50a770eac63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:02.894787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:02.912722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-8b2cffce-9f1a-4991-b088-e50a770eac63 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:02.914601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8b2cffce-9f1a-4991-b088-e50a770eac63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.974710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1017/2015] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:54:02.985205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cffce-9f1a-4991-b088-e50a770eac63 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:02.986786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8b2cffce-9f1a-4991-b088-e50a770eac63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.987861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bceff84a32f4e90a6407d5bcdda9946', 'request_id': 'req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.051412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cffce-9f1a-4991-b088-e50a770eac63 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:03.064199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bceff84a32f4e90a6407d5bcdda9946', 'request_id': 'req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.071527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bceff84a32f4e90a6407d5bcdda9946', 'request_id': 'req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.083068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] The token cache is being invalidate because project 4bceff84a32f4e90a6407d5bcdda9946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:03.083351 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.084007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.089579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 999/2016] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:02 2022] DELETE /identity/v3/projects/4bceff84a32f4e90a6407d5bcdda9946 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:03.089579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:03.089579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:03.103524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efbb08-c5f4-4d06-9896-7da4cba08e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:03.108550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efbb08-c5f4-4d06-9896-7da4cba08e38 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:03.108550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efbb08-c5f4-4d06-9896-7da4cba08e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:03.108550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efbb08-c5f4-4d06-9896-7da4cba08e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:03.108550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d6efbb08-c5f4-4d06-9896-7da4cba08e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:03.157260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9eb877-0a8f-4ac1-be4c-62b72251cda7 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:03.162645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:03.166404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:03.166404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:03.166404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] PATH_INFO: `/v3/projects/03ecd1067eb14f1e85a681090bc8fd20` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:03.166404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ecd1067eb14f1e85a681090bc8fd20)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:03.168432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:03.171313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efbb08-c5f4-4d06-9896-7da4cba08e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:03.172755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efbb08-c5f4-4d06-9896-7da4cba08e38 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:03.183693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1018/2017] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:54:03.187963 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4e104-62cc-4ff2-b0c9-33f2c290f2cb None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:03.188387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4e104-62cc-4ff2-b0c9-33f2c290f2cb None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:03.189001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4e104-62cc-4ff2-b0c9-33f2c290f2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:03.212473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6eb4e104-62cc-4ff2-b0c9-33f2c290f2cb None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:03.213689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6eb4e104-62cc-4ff2-b0c9-33f2c290f2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:03.221656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1019/2018] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:03.230415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de88452-ecc3-47f1-bc11-1eb2a4a4b19d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:03.230740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de88452-ecc3-47f1-bc11-1eb2a4a4b19d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:03.231018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de88452-ecc3-47f1-bc11-1eb2a4a4b19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:03.250705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9de88452-ecc3-47f1-bc11-1eb2a4a4b19d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:03.251933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9de88452-ecc3-47f1-bc11-1eb2a4a4b19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:03.260982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1020/2019] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:54:03.265475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de88452-ecc3-47f1-bc11-1eb2a4a4b19d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:03.276234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de88452-ecc3-47f1-bc11-1eb2a4a4b19d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:03.290918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ecd1067eb14f1e85a681090bc8fd20', 'request_id': 'req-56b77092-46ed-40a4-b627-ba1cc1c9eb90'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.334436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b33f-aaf4-48e8-9350-6041e4ebf198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:03.335086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b33f-aaf4-48e8-9350-6041e4ebf198 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:03.336040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b33f-aaf4-48e8-9350-6041e4ebf198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:03.336376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b33f-aaf4-48e8-9350-6041e4ebf198 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:03.337118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8b33f-aaf4-48e8-9350-6041e4ebf198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:54:03.337426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8b33f-aaf4-48e8-9350-6041e4ebf198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:03.339058 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8b33f-aaf4-48e8-9350-6041e4ebf198 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:03.350283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1021/2020] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:54:03.361044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b33f-aaf4-48e8-9350-6041e4ebf198 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:03.370656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b33f-aaf4-48e8-9350-6041e4ebf198 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:03.375415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84725586-d969-4e76-adaa-99b688af3ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:03.376000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84725586-d969-4e76-adaa-99b688af3ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:03.376289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84725586-d969-4e76-adaa-99b688af3ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:03.376561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84725586-d969-4e76-adaa-99b688af3ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:03.377206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84725586-d969-4e76-adaa-99b688af3ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:03.379144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84725586-d969-4e76-adaa-99b688af3ee3 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:03.441923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ecd1067eb14f1e85a681090bc8fd20', 'request_id': 'req-56b77092-46ed-40a4-b627-ba1cc1c9eb90'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.451131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ecd1067eb14f1e85a681090bc8fd20', 'request_id': 'req-56b77092-46ed-40a4-b627-ba1cc1c9eb90'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.464073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] The token cache is being invalidate because project 03ecd1067eb14f1e85a681090bc8fd20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:03.464374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.465194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.488214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1000/2021] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:03 2022] DELETE /identity/v3/projects/03ecd1067eb14f1e85a681090bc8fd20 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:03.488214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:03.488214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:03.500594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b77092-46ed-40a4-b627-ba1cc1c9eb90 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:03.509814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:03.510661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:03.511040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:03.511366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] PATH_INFO: `/v3/users/de80dbfffc324873a09da0104c4a992f` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:03.528637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1022/2022] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:03 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:03.528840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=de80dbfffc324873a09da0104c4a992f)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:03.531051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84725586-d969-4e76-adaa-99b688af3ee3 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:03.532785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:03.533444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-84725586-d969-4e76-adaa-99b688af3ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:03.602855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:03.628538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84725586-d969-4e76-adaa-99b688af3ee3 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:03.634569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fd002-67ec-4a11-8e25-9474a8c850d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:03.637175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fd002-67ec-4a11-8e25-9474a8c850d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:03.637175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fd002-67ec-4a11-8e25-9474a8c850d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:03.637175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fd002-67ec-4a11-8e25-9474a8c850d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:03.637175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45fd002-67ec-4a11-8e25-9474a8c850d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:03.639579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45fd002-67ec-4a11-8e25-9474a8c850d1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:03.772660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1023/2023] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:03 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:03.772660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fd002-67ec-4a11-8e25-9474a8c850d1 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:03.778451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f45fd002-67ec-4a11-8e25-9474a8c850d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:03.799671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fd002-67ec-4a11-8e25-9474a8c850d1 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:03.808088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:03.809149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:03.809677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:03.810122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] PATH_INFO: `/v3/projects/7a23fa82f0ac49d99b7d29fee241bad4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:03.812833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a23fa82f0ac49d99b7d29fee241bad4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:03.814927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:03.847387 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de80dbfffc324873a09da0104c4a992f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.923841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a23fa82f0ac49d99b7d29fee241bad4', 'request_id': 'req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.929888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a23fa82f0ac49d99b7d29fee241bad4', 'request_id': 'req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.933785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de80dbfffc324873a09da0104c4a992f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.948770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de80dbfffc324873a09da0104c4a992f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.956853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de80dbfffc324873a09da0104c4a992f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.957651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de80dbfffc324873a09da0104c4a992f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:03.964708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1001/2024] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:03 2022] DELETE /identity/v3/users/de80dbfffc324873a09da0104c4a992f => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:03.966085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:03.967619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:04.009904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a23fa82f0ac49d99b7d29fee241bad4', 'request_id': 'req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.042489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] The token cache is being invalidate because project 7a23fa82f0ac49d99b7d29fee241bad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:04.042860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.043566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.047624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1024/2025] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:03 2022] DELETE /identity/v3/projects/7a23fa82f0ac49d99b7d29fee241bad4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:04.051798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:04.055844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:04.065071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e1dd7-9014-4a2b-a817-a7d0d5ab6bac admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:04.078897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b80abd-cc73-46da-a20b-695289ea5b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:04.080211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80abd-cc73-46da-a20b-695289ea5b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:04.080503 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80abd-cc73-46da-a20b-695289ea5b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:04.080917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b80abd-cc73-46da-a20b-695289ea5b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:04.081460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b80abd-cc73-46da-a20b-695289ea5b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:04.084800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b80abd-cc73-46da-a20b-695289ea5b17 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:04.109119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1002/2026] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:04.115758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b80abd-cc73-46da-a20b-695289ea5b17 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:04.120486 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-82b80abd-cc73-46da-a20b-695289ea5b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:04.140076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b80abd-cc73-46da-a20b-695289ea5b17 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:04.149170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4fc80-0e5f-4bcc-b549-11b77e2b221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:04.150052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4fc80-0e5f-4bcc-b549-11b77e2b221f admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:04.150516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7b7cc-6dfe-43d3-b637-d3ed9c504395 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:04.150626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4fc80-0e5f-4bcc-b549-11b77e2b221f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:04.151028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4fc80-0e5f-4bcc-b549-11b77e2b221f admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:04.151842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4fc80-0e5f-4bcc-b549-11b77e2b221f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:04.154044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4fc80-0e5f-4bcc-b549-11b77e2b221f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:04.157429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:04.158321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:04.158799 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:04.159237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] PATH_INFO: `/v3/users/9a79a04d1ace4d36bc11d663974f6d18` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:04.160583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a79a04d1ace4d36bc11d663974f6d18)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:04.162608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:04.182873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1003/2027] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:54:04.191761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4fc80-0e5f-4bcc-b549-11b77e2b221f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:04.213350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4fc80-0e5f-4bcc-b549-11b77e2b221f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:04.221440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074163db-93e9-4ccf-9206-da103ca33d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:04.222753 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074163db-93e9-4ccf-9206-da103ca33d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:04.223322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074163db-93e9-4ccf-9206-da103ca33d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:04.223668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074163db-93e9-4ccf-9206-da103ca33d0d admin admin] PATH_INFO: `/v3/projects/5d84c2db19c24ef49d5012dd6c28cb7b/users/c662a5d21958400daa83733a93ca4c11/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:04.247830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074163db-93e9-4ccf-9206-da103ca33d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d84c2db19c24ef49d5012dd6c28cb7b, user_id=c662a5d21958400daa83733a93ca4c11, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:04.251728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074163db-93e9-4ccf-9206-da103ca33d0d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:04.376514 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1004/2028] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:04 2022] PUT /identity/v3/projects/5d84c2db19c24ef49d5012dd6c28cb7b/users/c662a5d21958400daa83733a93ca4c11/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:04.382472 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106307d4-2d0b-4346-aa27-af8ab554e03f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:04.383047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106307d4-2d0b-4346-aa27-af8ab554e03f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:04.383464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106307d4-2d0b-4346-aa27-af8ab554e03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:04.408736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-106307d4-2d0b-4346-aa27-af8ab554e03f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:04.410088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-106307d4-2d0b-4346-aa27-af8ab554e03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:04.436946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a79a04d1ace4d36bc11d663974f6d18'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.437974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a79a04d1ace4d36bc11d663974f6d18'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.482687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a79a04d1ace4d36bc11d663974f6d18'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.483036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1005/2029] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:54:04.497817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106307d4-2d0b-4346-aa27-af8ab554e03f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:04.499818 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-106307d4-2d0b-4346-aa27-af8ab554e03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:04.507735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a79a04d1ace4d36bc11d663974f6d18'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.532227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a79a04d1ace4d36bc11d663974f6d18'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.551303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1025/2030] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:04 2022] DELETE /identity/v3/users/9a79a04d1ace4d36bc11d663974f6d18 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:54:04.604814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106307d4-2d0b-4346-aa27-af8ab554e03f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:04.664948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa85a0-3b58-4b42-975e-e0d786771060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:04.665730 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa85a0-3b58-4b42-975e-e0d786771060 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:04.665862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa85a0-3b58-4b42-975e-e0d786771060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:04.666114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa85a0-3b58-4b42-975e-e0d786771060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:04.672791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:04.689250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:04.692038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:04.713473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:04.769278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa85a0-3b58-4b42-975e-e0d786771060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:04.770865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa85a0-3b58-4b42-975e-e0d786771060 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:04.785755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1006/2031] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:54:04.786007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443cded-3e25-46a7-9e84-ba85806e0c95 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:04.791786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:04.792741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:04.793088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:04.793438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] PATH_INFO: `/v3/projects/671dd7be873547c888bb7f7a341c93c4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:04.795161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=671dd7be873547c888bb7f7a341c93c4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:04.798260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:04.952272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '671dd7be873547c888bb7f7a341c93c4', 'request_id': 'req-dfeff6be-6fb6-44f9-ba59-1f9597cba395'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.959102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '671dd7be873547c888bb7f7a341c93c4', 'request_id': 'req-dfeff6be-6fb6-44f9-ba59-1f9597cba395'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.053617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '671dd7be873547c888bb7f7a341c93c4', 'request_id': 'req-dfeff6be-6fb6-44f9-ba59-1f9597cba395'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.070444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] The token cache is being invalidate because project 671dd7be873547c888bb7f7a341c93c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:05.070994 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.071788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-dfeff6be-6fb6-44f9-ba59-1f9597cba395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.073627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1026/2032] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:04 2022] DELETE /identity/v3/projects/671dd7be873547c888bb7f7a341c93c4 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:05.078536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9ab43-3cb0-4782-b8bd-132f77cb1f65 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.079041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9ab43-3cb0-4782-b8bd-132f77cb1f65 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.079142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9ab43-3cb0-4782-b8bd-132f77cb1f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.102892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-85b9ab43-3cb0-4782-b8bd-132f77cb1f65 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:05.104472 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-85b9ab43-3cb0-4782-b8bd-132f77cb1f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.174275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1007/2033] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:05.187899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8b531-309d-4376-8a4b-995bdd28e78b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.187899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8b531-309d-4376-8a4b-995bdd28e78b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.187899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8b531-309d-4376-8a4b-995bdd28e78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.191205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9ab43-3cb0-4782-b8bd-132f77cb1f65 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.193423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-85b9ab43-3cb0-4782-b8bd-132f77cb1f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.210334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9ab43-3cb0-4782-b8bd-132f77cb1f65 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.213044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.213557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d0e8b531-309d-4376-8a4b-995bdd28e78b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:05.213718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.213984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.214266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] PATH_INFO: `/v3/projects/80d05b42e73441ac95b2e5bfd6722758` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.214847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d0e8b531-309d-4376-8a4b-995bdd28e78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.215885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d05b42e73441ac95b2e5bfd6722758)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.217906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.230330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1027/2034] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:05.231658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8b531-309d-4376-8a4b-995bdd28e78b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.243506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8b531-309d-4376-8a4b-995bdd28e78b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.248916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b932aa6-687d-4b6d-a9a1-80fe1508796e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.249549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932aa6-687d-4b6d-a9a1-80fe1508796e admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.249834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932aa6-687d-4b6d-a9a1-80fe1508796e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.250733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932aa6-687d-4b6d-a9a1-80fe1508796e admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.251404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b932aa6-687d-4b6d-a9a1-80fe1508796e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:54:05.251742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b932aa6-687d-4b6d-a9a1-80fe1508796e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.253352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b932aa6-687d-4b6d-a9a1-80fe1508796e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.266449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1028/2035] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:54:05.271203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b932aa6-687d-4b6d-a9a1-80fe1508796e admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.282845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b932aa6-687d-4b6d-a9a1-80fe1508796e admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.287951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba769eb-8cdc-4c52-9f54-a68e26abd7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.289401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba769eb-8cdc-4c52-9f54-a68e26abd7f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.289684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba769eb-8cdc-4c52-9f54-a68e26abd7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.289967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba769eb-8cdc-4c52-9f54-a68e26abd7f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.290621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba769eb-8cdc-4c52-9f54-a68e26abd7f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.292596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba769eb-8cdc-4c52-9f54-a68e26abd7f4 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.325723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d05b42e73441ac95b2e5bfd6722758', 'request_id': 'req-151181b4-2474-4c37-824c-547875bea94d'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.387483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1029/2036] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:05.390024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d05b42e73441ac95b2e5bfd6722758', 'request_id': 'req-151181b4-2474-4c37-824c-547875bea94d'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.391496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba769eb-8cdc-4c52-9f54-a68e26abd7f4 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.398273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d05b42e73441ac95b2e5bfd6722758', 'request_id': 'req-151181b4-2474-4c37-824c-547875bea94d'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.411795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] The token cache is being invalidate because project 80d05b42e73441ac95b2e5bfd6722758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:05.412024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.412703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-151181b4-2474-4c37-824c-547875bea94d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.418344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1008/2037] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:05 2022] DELETE /identity/v3/projects/80d05b42e73441ac95b2e5bfd6722758 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:54:05.422706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33312237-31e4-4cb8-9251-f5886cddb638 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.422706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33312237-31e4-4cb8-9251-f5886cddb638 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.422706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33312237-31e4-4cb8-9251-f5886cddb638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.441591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-33312237-31e4-4cb8-9251-f5886cddb638 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:05.443102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-33312237-31e4-4cb8-9251-f5886cddb638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.469545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba769eb-8cdc-4c52-9f54-a68e26abd7f4 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.476902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f69634-40f8-48bc-8563-4514484b5ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.481675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f69634-40f8-48bc-8563-4514484b5ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.481675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f69634-40f8-48bc-8563-4514484b5ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.481675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f69634-40f8-48bc-8563-4514484b5ba4 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.481675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f69634-40f8-48bc-8563-4514484b5ba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.481675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f69634-40f8-48bc-8563-4514484b5ba4 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.518738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1009/2038] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:05.527134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9592809-8e69-4442-b836-96d1c5d4f034 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.527434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9592809-8e69-4442-b836-96d1c5d4f034 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.527705 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9592809-8e69-4442-b836-96d1c5d4f034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.548972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e9592809-8e69-4442-b836-96d1c5d4f034 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:05.554655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e9592809-8e69-4442-b836-96d1c5d4f034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.559482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1010/2039] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:05.563861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9592809-8e69-4442-b836-96d1c5d4f034 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.573315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9592809-8e69-4442-b836-96d1c5d4f034 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.577923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b289f-e545-49ec-95e2-1d9e9991c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.578532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b289f-e545-49ec-95e2-1d9e9991c767 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.578825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b289f-e545-49ec-95e2-1d9e9991c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.579099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b289f-e545-49ec-95e2-1d9e9991c767 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.579740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b289f-e545-49ec-95e2-1d9e9991c767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:54:05.580043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b289f-e545-49ec-95e2-1d9e9991c767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.581635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b289f-e545-49ec-95e2-1d9e9991c767 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.593536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1011/2040] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:54:05.597870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b289f-e545-49ec-95e2-1d9e9991c767 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.608429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b289f-e545-49ec-95e2-1d9e9991c767 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.613190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf1b8d-781e-4b3c-9837-15cab082b88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.613771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf1b8d-781e-4b3c-9837-15cab082b88e admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.614048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf1b8d-781e-4b3c-9837-15cab082b88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.614320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf1b8d-781e-4b3c-9837-15cab082b88e admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.614968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf1b8d-781e-4b3c-9837-15cab082b88e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.617481 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf1b8d-781e-4b3c-9837-15cab082b88e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.625535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1030/2041] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:05.630839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f69634-40f8-48bc-8563-4514484b5ba4 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.640946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-46f69634-40f8-48bc-8563-4514484b5ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.644921 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f69634-40f8-48bc-8563-4514484b5ba4 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.650118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e75d-c98f-4ea9-bfea-9be4556d617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.650765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e75d-c98f-4ea9-bfea-9be4556d617f admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.651049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e75d-c98f-4ea9-bfea-9be4556d617f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.651354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e75d-c98f-4ea9-bfea-9be4556d617f admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.652142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e75d-c98f-4ea9-bfea-9be4556d617f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.653751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e75d-c98f-4ea9-bfea-9be4556d617f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.664296 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1031/2042] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:54:05.668323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e75d-c98f-4ea9-bfea-9be4556d617f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.678232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e75d-c98f-4ea9-bfea-9be4556d617f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.683888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887645d3-2a37-44a5-a0ef-8181e80f269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.684504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887645d3-2a37-44a5-a0ef-8181e80f269e admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.684775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887645d3-2a37-44a5-a0ef-8181e80f269e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.685049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887645d3-2a37-44a5-a0ef-8181e80f269e admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.685697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887645d3-2a37-44a5-a0ef-8181e80f269e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.687338 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887645d3-2a37-44a5-a0ef-8181e80f269e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.703676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1032/2043] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:05.707852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887645d3-2a37-44a5-a0ef-8181e80f269e admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.718616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887645d3-2a37-44a5-a0ef-8181e80f269e admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.723421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761eab71-b9b0-4c6c-a1f2-58e1eef0c856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.724087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eab71-b9b0-4c6c-a1f2-58e1eef0c856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.724368 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eab71-b9b0-4c6c-a1f2-58e1eef0c856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.724642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eab71-b9b0-4c6c-a1f2-58e1eef0c856 admin admin] PATH_INFO: `/v3/projects/db6d0d7e6f2649f19912fb2a98c34634/users/38f6a3c38f4e4c86a45aa4890bcc0b9d/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.740414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761eab71-b9b0-4c6c-a1f2-58e1eef0c856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db6d0d7e6f2649f19912fb2a98c34634, user_id=38f6a3c38f4e4c86a45aa4890bcc0b9d, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.744068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761eab71-b9b0-4c6c-a1f2-58e1eef0c856 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.750541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1012/2044] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:05.753239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf1b8d-781e-4b3c-9837-15cab082b88e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.816831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf1b8d-781e-4b3c-9837-15cab082b88e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.822189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bcd71-4060-44e6-8627-8104162c2399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.822908 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bcd71-4060-44e6-8627-8104162c2399 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.823422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bcd71-4060-44e6-8627-8104162c2399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.823547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bcd71-4060-44e6-8627-8104162c2399 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.824121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bcd71-4060-44e6-8627-8104162c2399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.826586 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bcd71-4060-44e6-8627-8104162c2399 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.827019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1033/2045] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:05 2022] PUT /identity/v3/projects/db6d0d7e6f2649f19912fb2a98c34634/users/38f6a3c38f4e4c86a45aa4890bcc0b9d/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:54:05.838623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b432f-28be-4850-8625-01b985c80dd0 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.843084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b432f-28be-4850-8625-01b985c80dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.843084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b432f-28be-4850-8625-01b985c80dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.868531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-3b6b432f-28be-4850-8625-01b985c80dd0 None None] MFA Rules not processed for user `38f6a3c38f4e4c86a45aa4890bcc0b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:05.869833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3b6b432f-28be-4850-8625-01b985c80dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.893955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:05.940842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1034/2046] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:54:05.949731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1013/2047] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:05.950140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b432f-28be-4850-8625-01b985c80dd0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.950825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bcd71-4060-44e6-8627-8104162c2399 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.951637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3b6b432f-28be-4850-8625-01b985c80dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.959801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:05.962236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bcd71-4060-44e6-8627-8104162c2399 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.014470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b432f-28be-4850-8625-01b985c80dd0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.019592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b26da-e590-42c7-90d2-f5255f7438c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.020256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b26da-e590-42c7-90d2-f5255f7438c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.020467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b26da-e590-42c7-90d2-f5255f7438c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.020729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b26da-e590-42c7-90d2-f5255f7438c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.026699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67303ede-2de2-4af5-8a28-13bebf5a24b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.027323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67303ede-2de2-4af5-8a28-13bebf5a24b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.027611 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67303ede-2de2-4af5-8a28-13bebf5a24b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.027897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67303ede-2de2-4af5-8a28-13bebf5a24b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.028620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67303ede-2de2-4af5-8a28-13bebf5a24b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.030260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67303ede-2de2-4af5-8a28-13bebf5a24b9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.031636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b26da-e590-42c7-90d2-f5255f7438c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.033393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b26da-e590-42c7-90d2-f5255f7438c3 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.042746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1014/2048] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:54:06.047634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1035/2049] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:06.047784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67303ede-2de2-4af5-8a28-13bebf5a24b9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.058765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67303ede-2de2-4af5-8a28-13bebf5a24b9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.064461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a3c85-e43b-4245-bc52-c8dfde92f4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.064461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a3c85-e43b-4245-bc52-c8dfde92f4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.064740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a3c85-e43b-4245-bc52-c8dfde92f4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.064825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a3c85-e43b-4245-bc52-c8dfde92f4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.065416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a3c85-e43b-4245-bc52-c8dfde92f4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.067059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a3c85-e43b-4245-bc52-c8dfde92f4b9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.078804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1015/2050] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:54:06.083206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b26da-e590-42c7-90d2-f5255f7438c3 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.098057 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b26da-e590-42c7-90d2-f5255f7438c3 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.104312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29537c86-b247-478e-bd72-173ae8f7cab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.105063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29537c86-b247-478e-bd72-173ae8f7cab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.105356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29537c86-b247-478e-bd72-173ae8f7cab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.105639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29537c86-b247-478e-bd72-173ae8f7cab0 admin admin] PATH_INFO: `/v3/projects/d168492e976c4cbab9824f0b8573c551/users/050cb963a37b40fc946863c1947e1195/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.121788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29537c86-b247-478e-bd72-173ae8f7cab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d168492e976c4cbab9824f0b8573c551, user_id=050cb963a37b40fc946863c1947e1195, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.124527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29537c86-b247-478e-bd72-173ae8f7cab0 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.215653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1036/2051] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:06 2022] PUT /identity/v3/projects/d168492e976c4cbab9824f0b8573c551/users/050cb963a37b40fc946863c1947e1195/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:06.228234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e31ec5-45b9-4115-aa07-df21723fa521 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.228663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e31ec5-45b9-4115-aa07-df21723fa521 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.229045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e31ec5-45b9-4115-aa07-df21723fa521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.264270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-50e31ec5-45b9-4115-aa07-df21723fa521 None None] MFA Rules not processed for user `050cb963a37b40fc946863c1947e1195`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:06.265751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-50e31ec5-45b9-4115-aa07-df21723fa521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:06.348979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1016/2052] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:06.356538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29537c86-b247-478e-bd72-173ae8f7cab0 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.442059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29537c86-b247-478e-bd72-173ae8f7cab0 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.447381 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8e630-521c-4aca-b7e4-c58f79864991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.448011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e630-521c-4aca-b7e4-c58f79864991 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.448295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e630-521c-4aca-b7e4-c58f79864991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.448569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e630-521c-4aca-b7e4-c58f79864991 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.449214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8e630-521c-4aca-b7e4-c58f79864991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.451181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8e630-521c-4aca-b7e4-c58f79864991 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.613859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1037/2053] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:06 2022] POST /identity/v3/projects => generated 377 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:06.619109 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e31ec5-45b9-4115-aa07-df21723fa521 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.637462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e31ec5-45b9-4115-aa07-df21723fa521 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.705375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170393f9-d2e5-453b-b206-47e54d5e6d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.706131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170393f9-d2e5-453b-b206-47e54d5e6d30 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.706496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170393f9-d2e5-453b-b206-47e54d5e6d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.706805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170393f9-d2e5-453b-b206-47e54d5e6d30 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.707454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170393f9-d2e5-453b-b206-47e54d5e6d30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.709511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170393f9-d2e5-453b-b206-47e54d5e6d30 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.758205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:06.785193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1017/2054] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:06 2022] POST /identity/v3/users => generated 295 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:06.785332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8e630-521c-4aca-b7e4-c58f79864991 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.796855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8e630-521c-4aca-b7e4-c58f79864991 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.808132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef508c99-e572-42f8-a191-de0159226077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.809500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef508c99-e572-42f8-a191-de0159226077 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.809937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef508c99-e572-42f8-a191-de0159226077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.810362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef508c99-e572-42f8-a191-de0159226077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.811234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef508c99-e572-42f8-a191-de0159226077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.813167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef508c99-e572-42f8-a191-de0159226077 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.827335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1038/2055] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:06.831156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170393f9-d2e5-453b-b206-47e54d5e6d30 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.840545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:06.844938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170393f9-d2e5-453b-b206-47e54d5e6d30 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.851365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c65fb-4bea-49ae-8321-5ad9089fd75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.852046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c65fb-4bea-49ae-8321-5ad9089fd75c admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.852404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c65fb-4bea-49ae-8321-5ad9089fd75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.852789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c65fb-4bea-49ae-8321-5ad9089fd75c admin admin] PATH_INFO: `/v3/projects/b4ea591f2c734fceaae6e62da6d5df66/users/2f1b1b56d42e48c3b873dcd7b115be94/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.875536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c65fb-4bea-49ae-8321-5ad9089fd75c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ea591f2c734fceaae6e62da6d5df66, user_id=2f1b1b56d42e48c3b873dcd7b115be94, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.878821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c65fb-4bea-49ae-8321-5ad9089fd75c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.948552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1018/2056] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:06 2022] PUT /identity/v3/projects/b4ea591f2c734fceaae6e62da6d5df66/users/2f1b1b56d42e48c3b873dcd7b115be94/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:06.952412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef508c99-e572-42f8-a191-de0159226077 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.965839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef508c99-e572-42f8-a191-de0159226077 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:07.021709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:07.066875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b836dfa-dd12-4c13-b9f3-83fa555e6709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:07.067831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b836dfa-dd12-4c13-b9f3-83fa555e6709 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:07.068516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b836dfa-dd12-4c13-b9f3-83fa555e6709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:07.069249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b836dfa-dd12-4c13-b9f3-83fa555e6709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:07.070139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b836dfa-dd12-4c13-b9f3-83fa555e6709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:07.072022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b836dfa-dd12-4c13-b9f3-83fa555e6709 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:07.096456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1039/2057] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:54:07.101235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c65fb-4bea-49ae-8321-5ad9089fd75c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:07.118042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c65fb-4bea-49ae-8321-5ad9089fd75c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:07.127192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff80ad5-a3ae-4cd6-8fb6-5b567f4870e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:07.128777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff80ad5-a3ae-4cd6-8fb6-5b567f4870e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:07.129602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff80ad5-a3ae-4cd6-8fb6-5b567f4870e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:07.130791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff80ad5-a3ae-4cd6-8fb6-5b567f4870e6 admin admin] PATH_INFO: `/v3/projects/b4ea591f2c734fceaae6e62da6d5df66/users/2f1b1b56d42e48c3b873dcd7b115be94/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:07.134832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff80ad5-a3ae-4cd6-8fb6-5b567f4870e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ea591f2c734fceaae6e62da6d5df66, user_id=2f1b1b56d42e48c3b873dcd7b115be94, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:07.137894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff80ad5-a3ae-4cd6-8fb6-5b567f4870e6 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:07.151611 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1019/2058] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:07 2022] PUT /identity/v3/projects/b4ea591f2c734fceaae6e62da6d5df66/users/2f1b1b56d42e48c3b873dcd7b115be94/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:07.163998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed19153-6efb-4179-8f2f-e1b88e01e719 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:07.164363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed19153-6efb-4179-8f2f-e1b88e01e719 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:07.164714 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed19153-6efb-4179-8f2f-e1b88e01e719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:07.218095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-2ed19153-6efb-4179-8f2f-e1b88e01e719 None None] MFA Rules not processed for user `2f1b1b56d42e48c3b873dcd7b115be94`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:07.219783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2ed19153-6efb-4179-8f2f-e1b88e01e719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:07.393088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1040/2059] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:07.408210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff80ad5-a3ae-4cd6-8fb6-5b567f4870e6 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:07.572578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff80ad5-a3ae-4cd6-8fb6-5b567f4870e6 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:07.580908 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f48124-343c-4508-8cbb-b8b35c364197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:07.581536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f48124-343c-4508-8cbb-b8b35c364197 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:07.581825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f48124-343c-4508-8cbb-b8b35c364197 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:07.582099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f48124-343c-4508-8cbb-b8b35c364197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:07.769701 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f48124-343c-4508-8cbb-b8b35c364197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:07.778361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f48124-343c-4508-8cbb-b8b35c364197 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:07.782601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:07.806829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1020/2060] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:08.524642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:09.504731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:11.919832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14c22-7e58-471f-a9e1-f3b0bc42a15b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:11.920543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14c22-7e58-471f-a9e1-f3b0bc42a15b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:11.920543 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14c22-7e58-471f-a9e1-f3b0bc42a15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:11.951591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-aeb14c22-7e58-471f-a9e1-f3b0bc42a15b None None] MFA Rules not processed for user `c662a5d21958400daa83733a93ca4c11`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:11.951591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-aeb14c22-7e58-471f-a9e1-f3b0bc42a15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:11.955182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:12.027920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1041/2061] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:12.032986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f48124-343c-4508-8cbb-b8b35c364197 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:12.034579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-22f48124-343c-4508-8cbb-b8b35c364197 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:12.110380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f48124-343c-4508-8cbb-b8b35c364197 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:12.115685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5896d-0f6a-44f3-89af-87d51bd44d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:12.116444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5896d-0f6a-44f3-89af-87d51bd44d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:12.116580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5896d-0f6a-44f3-89af-87d51bd44d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:12.116854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5896d-0f6a-44f3-89af-87d51bd44d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:12.117494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5896d-0f6a-44f3-89af-87d51bd44d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:12.119487 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5896d-0f6a-44f3-89af-87d51bd44d18 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:12.129738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:12.305922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1021/2062] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:12 2022] POST /identity/v3/projects => generated 383 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:12.310021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb14c22-7e58-471f-a9e1-f3b0bc42a15b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:12.322484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb14c22-7e58-471f-a9e1-f3b0bc42a15b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:12.386696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabeb30-4503-4af1-8610-386c3d6c4fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:12.386696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabeb30-4503-4af1-8610-386c3d6c4fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:12.386867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabeb30-4503-4af1-8610-386c3d6c4fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:12.387099 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabeb30-4503-4af1-8610-386c3d6c4fe1 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:12.387736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabeb30-4503-4af1-8610-386c3d6c4fe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:12.389665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabeb30-4503-4af1-8610-386c3d6c4fe1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:12.532991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1042/2063] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:12 2022] POST /identity/v3/users => generated 298 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:12.537084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5896d-0f6a-44f3-89af-87d51bd44d18 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:12.547794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5896d-0f6a-44f3-89af-87d51bd44d18 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:12.552444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da1c5c-f2f7-4df4-97c0-02cac26b2b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:12.553573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1c5c-f2f7-4df4-97c0-02cac26b2b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:12.553939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1c5c-f2f7-4df4-97c0-02cac26b2b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:12.554223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1c5c-f2f7-4df4-97c0-02cac26b2b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:12.555185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da1c5c-f2f7-4df4-97c0-02cac26b2b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:12.556661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da1c5c-f2f7-4df4-97c0-02cac26b2b77 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:12.567846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1022/2064] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:12.572973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabeb30-4503-4af1-8610-386c3d6c4fe1 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:12.583399 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabeb30-4503-4af1-8610-386c3d6c4fe1 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:12.588333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2355da-bdb3-4bed-a35c-1e95ea28f7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:12.589001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2355da-bdb3-4bed-a35c-1e95ea28f7ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:12.589293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2355da-bdb3-4bed-a35c-1e95ea28f7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:12.589595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2355da-bdb3-4bed-a35c-1e95ea28f7ad admin admin] PATH_INFO: `/v3/projects/8e7cb7f74f9e4efcbfa816fe442475f8/users/5c2f361c4940443986148753ff4f27aa/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:12.604917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2355da-bdb3-4bed-a35c-1e95ea28f7ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7cb7f74f9e4efcbfa816fe442475f8, user_id=5c2f361c4940443986148753ff4f27aa, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:12.607681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2355da-bdb3-4bed-a35c-1e95ea28f7ad admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:12.708552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1043/2065] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:12 2022] PUT /identity/v3/projects/8e7cb7f74f9e4efcbfa816fe442475f8/users/5c2f361c4940443986148753ff4f27aa/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:12.709456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da1c5c-f2f7-4df4-97c0-02cac26b2b77 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:12.720596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da1c5c-f2f7-4df4-97c0-02cac26b2b77 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:12.789343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045e62-7f23-4c48-accf-0b28c8efd6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:12.789967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045e62-7f23-4c48-accf-0b28c8efd6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:12.790250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045e62-7f23-4c48-accf-0b28c8efd6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:12.790551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045e62-7f23-4c48-accf-0b28c8efd6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:12.791226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045e62-7f23-4c48-accf-0b28c8efd6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:12.792843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045e62-7f23-4c48-accf-0b28c8efd6fc admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:12.804736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1023/2066] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:12.808892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2355da-bdb3-4bed-a35c-1e95ea28f7ad admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:12.820562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2355da-bdb3-4bed-a35c-1e95ea28f7ad admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:12.825628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706489b-dc8b-4a52-8a38-ba2989c3f5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:12.826312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706489b-dc8b-4a52-8a38-ba2989c3f5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:12.826653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706489b-dc8b-4a52-8a38-ba2989c3f5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:12.826935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706489b-dc8b-4a52-8a38-ba2989c3f5f5 admin admin] PATH_INFO: `/v3/projects/8e7cb7f74f9e4efcbfa816fe442475f8/users/5c2f361c4940443986148753ff4f27aa/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:12.829951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706489b-dc8b-4a52-8a38-ba2989c3f5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7cb7f74f9e4efcbfa816fe442475f8, user_id=5c2f361c4940443986148753ff4f27aa, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:12.832676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706489b-dc8b-4a52-8a38-ba2989c3f5f5 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:12.853858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1044/2067] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:12 2022] PUT /identity/v3/projects/8e7cb7f74f9e4efcbfa816fe442475f8/users/5c2f361c4940443986148753ff4f27aa/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:54:14.762403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045e62-7f23-4c48-accf-0b28c8efd6fc admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:14.873960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045e62-7f23-4c48-accf-0b28c8efd6fc admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:14.958287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a744a9-59aa-4c63-8dd3-1c681819d8d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:14.959766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a744a9-59aa-4c63-8dd3-1c681819d8d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:14.960303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a744a9-59aa-4c63-8dd3-1c681819d8d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:14.960679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a744a9-59aa-4c63-8dd3-1c681819d8d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:15.073615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:15.102110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a744a9-59aa-4c63-8dd3-1c681819d8d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:15.104427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a744a9-59aa-4c63-8dd3-1c681819d8d2 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:15.135668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1024/2068] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:20.216551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128d9f3-c4df-4a28-8f27-f88496b84c65 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:20.217068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128d9f3-c4df-4a28-8f27-f88496b84c65 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:20.221706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128d9f3-c4df-4a28-8f27-f88496b84c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:20.249112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-4128d9f3-c4df-4a28-8f27-f88496b84c65 None None] MFA Rules not processed for user `5c2f361c4940443986148753ff4f27aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:20.273311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4128d9f3-c4df-4a28-8f27-f88496b84c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:20.316982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:20.345235 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1045/2069] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:20.357201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a744a9-59aa-4c63-8dd3-1c681819d8d2 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:20.358747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d2a744a9-59aa-4c63-8dd3-1c681819d8d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:20.450497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a744a9-59aa-4c63-8dd3-1c681819d8d2 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:20.463422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d1639-9648-4d29-82d1-b17e5e61dc6e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ff3f53ddbb04eb8916b9ccfdb03dd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:20.464586 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d1639-9648-4d29-82d1-b17e5e61dc6e service glance] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:20.473430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d1639-9648-4d29-82d1-b17e5e61dc6e service glance] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:20.473430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d1639-9648-4d29-82d1-b17e5e61dc6e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:20.532505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d1639-9648-4d29-82d1-b17e5e61dc6e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:20.534207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d1639-9648-4d29-82d1-b17e5e61dc6e service glance] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:20.545359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1025/2070] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:20.857492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3bb99-b336-4d86-9209-78edc23d9ac4 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:20.857782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3bb99-b336-4d86-9209-78edc23d9ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:20.858060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3bb99-b336-4d86-9209-78edc23d9ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:20.882572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-1ab3bb99-b336-4d86-9209-78edc23d9ac4 None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:20.882572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1ab3bb99-b336-4d86-9209-78edc23d9ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:20.967333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1046/2071] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:54:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:21.011308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d1639-9648-4d29-82d1-b17e5e61dc6e service glance] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:21.012959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a38d1639-9648-4d29-82d1-b17e5e61dc6e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:21.109432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d1639-9648-4d29-82d1-b17e5e61dc6e service glance] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:21.116615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695c031-ccc1-4aec-8e0d-1d4c6d77f937 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:21.117296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695c031-ccc1-4aec-8e0d-1d4c6d77f937 service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:21.117588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695c031-ccc1-4aec-8e0d-1d4c6d77f937 service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:21.117866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695c031-ccc1-4aec-8e0d-1d4c6d77f937 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:21.148034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695c031-ccc1-4aec-8e0d-1d4c6d77f937 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:21.150145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695c031-ccc1-4aec-8e0d-1d4c6d77f937 service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:21.173096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1026/2072] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:54:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:54:25.795325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad0769-373a-4f9a-9010-6a4cf39f8279 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:25.795325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad0769-373a-4f9a-9010-6a4cf39f8279 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:25.795325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad0769-373a-4f9a-9010-6a4cf39f8279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:25.814300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-1aad0769-373a-4f9a-9010-6a4cf39f8279 None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:25.815667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1aad0769-373a-4f9a-9010-6a4cf39f8279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:25.823402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1047/2073] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:54:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:25.838868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695c031-ccc1-4aec-8e0d-1d4c6d77f937 service swift] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:25.852048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695c031-ccc1-4aec-8e0d-1d4c6d77f937 service swift] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:25.860281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116187c-419a-495f-b240-6b6545017b7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:25.861153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116187c-419a-495f-b240-6b6545017b7c service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:25.861655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116187c-419a-495f-b240-6b6545017b7c service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:25.862100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116187c-419a-495f-b240-6b6545017b7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:25.876503 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116187c-419a-495f-b240-6b6545017b7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:25.878155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116187c-419a-495f-b240-6b6545017b7c service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:25.899949 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1027/2074] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:54:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:54:27.532618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2f853-17b3-4221-a53a-149e18e36415 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:27.535048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2f853-17b3-4221-a53a-149e18e36415 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:27.535048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2f853-17b3-4221-a53a-149e18e36415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:27.558539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-95b2f853-17b3-4221-a53a-149e18e36415 None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:27.559795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-95b2f853-17b3-4221-a53a-149e18e36415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:27.573586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1048/2075] 10.222.0.82 () {58 vars in 967 bytes} [Wed Aug 24 03:54:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:27.588987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116187c-419a-495f-b240-6b6545017b7c service swift] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:27.607579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116187c-419a-495f-b240-6b6545017b7c service swift] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:27.626993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55e9ba-ae7c-47f3-beaf-e4dd75fe8f33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:27.626993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55e9ba-ae7c-47f3-beaf-e4dd75fe8f33 service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:27.626993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55e9ba-ae7c-47f3-beaf-e4dd75fe8f33 service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:27.626993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55e9ba-ae7c-47f3-beaf-e4dd75fe8f33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:27.635374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55e9ba-ae7c-47f3-beaf-e4dd75fe8f33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:27.637114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55e9ba-ae7c-47f3-beaf-e4dd75fe8f33 service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:27.643091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:27.650691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1028/2076] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:54:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:54:29.480038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ad9d-916e-4659-ae56-f33c55323c2f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:29.480535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ad9d-916e-4659-ae56-f33c55323c2f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:29.480639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ad9d-916e-4659-ae56-f33c55323c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:29.499898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-8470ad9d-916e-4659-ae56-f33c55323c2f None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:29.501097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8470ad9d-916e-4659-ae56-f33c55323c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:29.509068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1049/2077] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:54:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:29.522491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55e9ba-ae7c-47f3-beaf-e4dd75fe8f33 service swift] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:29.533589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55e9ba-ae7c-47f3-beaf-e4dd75fe8f33 service swift] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:29.540688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426a93e-8689-49f2-9327-c846f2a037b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:29.541293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a93e-8689-49f2-9327-c846f2a037b5 service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:29.541615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a93e-8689-49f2-9327-c846f2a037b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:29.541895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a93e-8689-49f2-9327-c846f2a037b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:29.551025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426a93e-8689-49f2-9327-c846f2a037b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:29.552646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426a93e-8689-49f2-9327-c846f2a037b5 service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:29.560970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1029/2078] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:54:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:54:29.927557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ad9d-916e-4659-ae56-f33c55323c2f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:30.059924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ad9d-916e-4659-ae56-f33c55323c2f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:30.074594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:30.078066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:30.078671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:30.082124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] PATH_INFO: `/v3/users/38f6a3c38f4e4c86a45aa4890bcc0b9d` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:30.085282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f6a3c38f4e4c86a45aa4890bcc0b9d)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:30.090734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:30.345123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f6a3c38f4e4c86a45aa4890bcc0b9d'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:30.345863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f6a3c38f4e4c86a45aa4890bcc0b9d'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:30.463958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f6a3c38f4e4c86a45aa4890bcc0b9d'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:30.483476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f6a3c38f4e4c86a45aa4890bcc0b9d'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:30.504480 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:30.506591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652018-293a-4971-af34-c831037fdc7b None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:30.509941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652018-293a-4971-af34-c831037fdc7b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:30.509941 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8652018-293a-4971-af34-c831037fdc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:30.518729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7cd72476-547a-4e4b-b84b-4be8515061ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f6a3c38f4e4c86a45aa4890bcc0b9d'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:30.525741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1050/2079] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:29 2022] DELETE /identity/v3/users/38f6a3c38f4e4c86a45aa4890bcc0b9d => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:30.530175 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f3792-6841-45ee-bc7d-adc8852b5918 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:30.531899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f3792-6841-45ee-bc7d-adc8852b5918 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:30.534115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f3792-6841-45ee-bc7d-adc8852b5918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:30.538616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c8652018-293a-4971-af34-c831037fdc7b None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:30.539952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c8652018-293a-4971-af34-c831037fdc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:30.562655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-8a6f3792-6841-45ee-bc7d-adc8852b5918 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:30.564226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8a6f3792-6841-45ee-bc7d-adc8852b5918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:30.627957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1030/2080] 10.222.0.82 () {58 vars in 967 bytes} [Wed Aug 24 03:54:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:54:30.644615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8652018-293a-4971-af34-c831037fdc7b None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:30.646571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c8652018-293a-4971-af34-c831037fdc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:30.653464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1051/2081] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:30.668228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f3792-6841-45ee-bc7d-adc8852b5918 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:30.670962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8a6f3792-6841-45ee-bc7d-adc8852b5918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:30.728211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8652018-293a-4971-af34-c831037fdc7b None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:30.734131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3a025-ba4f-4358-82a0-1ab36d3bfa39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:30.734858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3a025-ba4f-4358-82a0-1ab36d3bfa39 service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:30.735363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3a025-ba4f-4358-82a0-1ab36d3bfa39 service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:30.735741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3a025-ba4f-4358-82a0-1ab36d3bfa39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:30.742292 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f3792-6841-45ee-bc7d-adc8852b5918 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:30.746152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3a025-ba4f-4358-82a0-1ab36d3bfa39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:30.746273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d9c1-56b5-43e2-840c-a9728c2937c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:30.747176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d9c1-56b5-43e2-840c-a9728c2937c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:30.747461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d9c1-56b5-43e2-840c-a9728c2937c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:30.747743 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d9c1-56b5-43e2-840c-a9728c2937c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:30.748368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3a025-ba4f-4358-82a0-1ab36d3bfa39 service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:30.761260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d9c1-56b5-43e2-840c-a9728c2937c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:30.764099 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d9c1-56b5-43e2-840c-a9728c2937c8 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:30.764349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1031/2082] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:54:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:54:30.776976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1052/2083] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:54:31.351839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3a025-ba4f-4358-82a0-1ab36d3bfa39 service swift] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:31.351839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-13e3a025-ba4f-4358-82a0-1ab36d3bfa39 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:31.363037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3a025-ba4f-4358-82a0-1ab36d3bfa39 service swift] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:31.376830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:31.377636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:31.378544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:31.378544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] PATH_INFO: `/v3/projects/db6d0d7e6f2649f19912fb2a98c34634` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:31.380195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db6d0d7e6f2649f19912fb2a98c34634)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:31.392884 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:31.619919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db6d0d7e6f2649f19912fb2a98c34634', 'request_id': 'req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:31.701348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db6d0d7e6f2649f19912fb2a98c34634', 'request_id': 'req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:31.711486 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db6d0d7e6f2649f19912fb2a98c34634', 'request_id': 'req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:31.729240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] The token cache is being invalidate because project db6d0d7e6f2649f19912fb2a98c34634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:31.729870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:31.732743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:31.735111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1032/2084] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:31 2022] DELETE /identity/v3/projects/db6d0d7e6f2649f19912fb2a98c34634 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:35.261290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a327fc-f7e2-498e-a60c-8579dc77adeb None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:35.261856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a327fc-f7e2-498e-a60c-8579dc77adeb None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:35.261937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a327fc-f7e2-498e-a60c-8579dc77adeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:35.283333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-75a327fc-f7e2-498e-a60c-8579dc77adeb None None] MFA Rules not processed for user `c662a5d21958400daa83733a93ca4c11`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:35.288181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-75a327fc-f7e2-498e-a60c-8579dc77adeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:35.398542 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1053/2085] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:37.497502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:37.499342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:37.506665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:37.602011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3914bf-d0a8-493c-91ac-6e62f98b6b2e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:37.608482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:37.609157 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:37.609522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:37.609811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] PATH_INFO: `/v3/users/a561c97996cc4677823df848773232e6` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:37.611044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a561c97996cc4677823df848773232e6)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:37.613102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:37.870193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993fae8-36f6-4eb6-a3f1-bd4a02870b6d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:37.870734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993fae8-36f6-4eb6-a3f1-bd4a02870b6d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:37.870991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993fae8-36f6-4eb6-a3f1-bd4a02870b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:37.899734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6993fae8-36f6-4eb6-a3f1-bd4a02870b6d None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:37.900985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6993fae8-36f6-4eb6-a3f1-bd4a02870b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:37.937318 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:37.966045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1054/2086] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:54:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:37.987965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993fae8-36f6-4eb6-a3f1-bd4a02870b6d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:37.987965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6993fae8-36f6-4eb6-a3f1-bd4a02870b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:38.052995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993fae8-36f6-4eb6-a3f1-bd4a02870b6d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:38.058087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85318091-03f0-4567-9154-eb4d42e9e503 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:38.062305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a561c97996cc4677823df848773232e6'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:38.062468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85318091-03f0-4567-9154-eb4d42e9e503 service swift] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:38.062468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85318091-03f0-4567-9154-eb4d42e9e503 service swift] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:38.062468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85318091-03f0-4567-9154-eb4d42e9e503 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:38.068627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85318091-03f0-4567-9154-eb4d42e9e503 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:38.070256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85318091-03f0-4567-9154-eb4d42e9e503 service swift] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:38.078371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1055/2087] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:54:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:54:38.228205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a561c97996cc4677823df848773232e6'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:38.242815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a561c97996cc4677823df848773232e6'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:38.249646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a561c97996cc4677823df848773232e6'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:38.250356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a561c97996cc4677823df848773232e6'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:38.257692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1033/2088] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:37 2022] DELETE /identity/v3/users/a561c97996cc4677823df848773232e6 => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:38.262268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf63af-4b3e-43a2-a683-150047fc7ae8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:38.262600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf63af-4b3e-43a2-a683-150047fc7ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:38.262913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf63af-4b3e-43a2-a683-150047fc7ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:38.288051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5fdf63af-4b3e-43a2-a683-150047fc7ae8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:38.289319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5fdf63af-4b3e-43a2-a683-150047fc7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:38.364800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1056/2089] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:38.371573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:38.373134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:38.442698 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1920c3-1bd6-4bf3-b8e0-76605d5a7005 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:38.447333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4d35-5275-438a-98f2-deecbe364dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:38.448993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4d35-5275-438a-98f2-deecbe364dba service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:38.449553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4d35-5275-438a-98f2-deecbe364dba service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:38.449827 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4d35-5275-438a-98f2-deecbe364dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:38.460380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4d35-5275-438a-98f2-deecbe364dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:38.461899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4d35-5275-438a-98f2-deecbe364dba service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:38.491881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1034/2090] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:54:38.915239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf63af-4b3e-43a2-a683-150047fc7ae8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:38.926019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5fdf63af-4b3e-43a2-a683-150047fc7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:38.966380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf63af-4b3e-43a2-a683-150047fc7ae8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:38.977134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:38.977810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:38.978097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:38.978377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] PATH_INFO: `/v3/projects/82af5eacc6004249910b0e3cbd8b7f34` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:38.980116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82af5eacc6004249910b0e3cbd8b7f34)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:38.982106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:39.241676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82af5eacc6004249910b0e3cbd8b7f34', 'request_id': 'req-db8de813-62ba-4f34-be10-df06303c5980'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:39.253459 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82af5eacc6004249910b0e3cbd8b7f34', 'request_id': 'req-db8de813-62ba-4f34-be10-df06303c5980'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:39.540792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82af5eacc6004249910b0e3cbd8b7f34', 'request_id': 'req-db8de813-62ba-4f34-be10-df06303c5980'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:39.542285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:39.561062 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] The token cache is being invalidate because project 82af5eacc6004249910b0e3cbd8b7f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:39.561262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:39.562208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-db8de813-62ba-4f34-be10-df06303c5980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:39.579122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1057/2091] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:38 2022] DELETE /identity/v3/projects/82af5eacc6004249910b0e3cbd8b7f34 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:43.446497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cc772-7a16-4376-96da-d710b921a17b None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:43.448485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cc772-7a16-4376-96da-d710b921a17b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:43.448791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cc772-7a16-4376-96da-d710b921a17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:43.475747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b15cc772-7a16-4376-96da-d710b921a17b None None] MFA Rules not processed for user `e5582d10ebef459084fe287c18568f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:43.477155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b15cc772-7a16-4376-96da-d710b921a17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:43.501723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:43.573095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1035/2092] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:45.154969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36958c-eb6b-4e2b-bd3c-7559db06d777 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:45.156154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36958c-eb6b-4e2b-bd3c-7559db06d777 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:45.156590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36958c-eb6b-4e2b-bd3c-7559db06d777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:45.180146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-bd36958c-eb6b-4e2b-bd3c-7559db06d777 None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:45.181839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bd36958c-eb6b-4e2b-bd3c-7559db06d777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:45.264447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1058/2093] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:54:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:45.317230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cc772-7a16-4376-96da-d710b921a17b None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:45.319566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b15cc772-7a16-4376-96da-d710b921a17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:45.445711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cc772-7a16-4376-96da-d710b921a17b None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:45.451992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b6cdf-b201-4773-839c-3433816872a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:45.466675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b6cdf-b201-4773-839c-3433816872a0 service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:45.466675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b6cdf-b201-4773-839c-3433816872a0 service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:45.466675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b6cdf-b201-4773-839c-3433816872a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:45.466675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b6cdf-b201-4773-839c-3433816872a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:45.466675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b6cdf-b201-4773-839c-3433816872a0 service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:45.474990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1036/2094] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:54:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:54:50.054229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36958c-eb6b-4e2b-bd3c-7559db06d777 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:50.056938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bd36958c-eb6b-4e2b-bd3c-7559db06d777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:50.155666 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36958c-eb6b-4e2b-bd3c-7559db06d777 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:50.161545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:50.162365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:50.162700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:50.162987 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] PATH_INFO: `/v3/users/050cb963a37b40fc946863c1947e1195` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:50.164173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050cb963a37b40fc946863c1947e1195)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:50.166310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:50.928877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050cb963a37b40fc946863c1947e1195'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:50.933044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050cb963a37b40fc946863c1947e1195'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:50.971193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab01401-64f0-4e53-a9c0-cc8b69800838 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:50.972600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab01401-64f0-4e53-a9c0-cc8b69800838 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:50.973046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab01401-64f0-4e53-a9c0-cc8b69800838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:51.025445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-0ab01401-64f0-4e53-a9c0-cc8b69800838 None None] MFA Rules not processed for user `547bce0d8133462c95bcb6f272776a27`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:51.025445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0ab01401-64f0-4e53-a9c0-cc8b69800838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:51.081879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050cb963a37b40fc946863c1947e1195'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:51.101516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050cb963a37b40fc946863c1947e1195'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:51.162375 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1c4e85a1-ec83-43f0-bd45-564ba8594401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050cb963a37b40fc946863c1947e1195'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:51.204805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1059/2095] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:50 2022] DELETE /identity/v3/users/050cb963a37b40fc946863c1947e1195 => generated 0 bytes in 1147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:51.206731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f977f76-584f-4ad5-8166-77f1cd4b7c4e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:51.207059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f977f76-584f-4ad5-8166-77f1cd4b7c4e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:51.207347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f977f76-584f-4ad5-8166-77f1cd4b7c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:51.224481 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1037/2096] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:51.277994 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-4f977f76-584f-4ad5-8166-77f1cd4b7c4e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:51.281228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4f977f76-584f-4ad5-8166-77f1cd4b7c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:51.320418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:51.446951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1060/2097] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:51.455279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab01401-64f0-4e53-a9c0-cc8b69800838 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:51.456865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0ab01401-64f0-4e53-a9c0-cc8b69800838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:51.549313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab01401-64f0-4e53-a9c0-cc8b69800838 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:51.574940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:51.652881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47368b2-891f-4c40-8427-cb89f4ca7445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:51.653554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47368b2-891f-4c40-8427-cb89f4ca7445 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:51.653850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47368b2-891f-4c40-8427-cb89f4ca7445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:51.654597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47368b2-891f-4c40-8427-cb89f4ca7445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:51.676915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47368b2-891f-4c40-8427-cb89f4ca7445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:51.676915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47368b2-891f-4c40-8427-cb89f4ca7445 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:51.699157 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1038/2098] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:54:52.253304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f977f76-584f-4ad5-8166-77f1cd4b7c4e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:52.256033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4f977f76-584f-4ad5-8166-77f1cd4b7c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:52.271358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f977f76-584f-4ad5-8166-77f1cd4b7c4e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:52.279122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:52.303683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:52.306638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:52.307728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] PATH_INFO: `/v3/projects/d168492e976c4cbab9824f0b8573c551` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:52.321192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d168492e976c4cbab9824f0b8573c551)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:52.329682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:52.541060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd168492e976c4cbab9824f0b8573c551', 'request_id': 'req-4bde4753-2fa3-4883-8a86-2c204ff174ed'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.555020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd168492e976c4cbab9824f0b8573c551', 'request_id': 'req-4bde4753-2fa3-4883-8a86-2c204ff174ed'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.653397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd168492e976c4cbab9824f0b8573c551', 'request_id': 'req-4bde4753-2fa3-4883-8a86-2c204ff174ed'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.669154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] The token cache is being invalidate because project d168492e976c4cbab9824f0b8573c551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:52.669513 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.675064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.678763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1061/2099] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:52 2022] DELETE /identity/v3/projects/d168492e976c4cbab9824f0b8573c551 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:52.689788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47368b2-891f-4c40-8427-cb89f4ca7445 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:52.706170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47368b2-891f-4c40-8427-cb89f4ca7445 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:52.920926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:52.922558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:52.923000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:52.923446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] PATH_INFO: `/v3/users/2f1b1b56d42e48c3b873dcd7b115be94` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:52.924800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1b1b56d42e48c3b873dcd7b115be94)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:52.926979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:53.163610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1b1b56d42e48c3b873dcd7b115be94'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.263860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1b1b56d42e48c3b873dcd7b115be94'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.276416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1b1b56d42e48c3b873dcd7b115be94'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.296526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1b1b56d42e48c3b873dcd7b115be94'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.298195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-abd06da1-bdcc-4dd9-87ef-e9670e6459fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1b1b56d42e48c3b873dcd7b115be94'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.304638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1039/2100] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:52 2022] DELETE /identity/v3/users/2f1b1b56d42e48c3b873dcd7b115be94 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:55.029260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:55.031130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:55.050438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde4753-2fa3-4883-8a86-2c204ff174ed admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:55.173357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:55.174312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:55.174804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:55.175080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] PATH_INFO: `/v3/projects/b4ea591f2c734fceaae6e62da6d5df66` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:55.176357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ea591f2c734fceaae6e62da6d5df66)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:55.178340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:55.332132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:55.335922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ea591f2c734fceaae6e62da6d5df66', 'request_id': 'req-400d544b-e791-4c38-8590-123e9dcd17e8'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:55.343037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ea591f2c734fceaae6e62da6d5df66', 'request_id': 'req-400d544b-e791-4c38-8590-123e9dcd17e8'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:55.441586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ea591f2c734fceaae6e62da6d5df66', 'request_id': 'req-400d544b-e791-4c38-8590-123e9dcd17e8'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:55.455893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] The token cache is being invalidate because project b4ea591f2c734fceaae6e62da6d5df66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:55.456546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:55.457413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-400d544b-e791-4c38-8590-123e9dcd17e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:55.459535 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1062/2101] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:55 2022] DELETE /identity/v3/projects/b4ea591f2c734fceaae6e62da6d5df66 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:55.464502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253fe3f-cf14-458c-86df-df4828bac7f6 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:55.465099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253fe3f-cf14-458c-86df-df4828bac7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:55.465268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253fe3f-cf14-458c-86df-df4828bac7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:55.487949 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-1253fe3f-cf14-458c-86df-df4828bac7f6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:55.489303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1253fe3f-cf14-458c-86df-df4828bac7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:55.533594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:55.555653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1040/2102] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:54:55.564067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856e07a-7284-4599-b100-c2a8677c1cee None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:55.565428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856e07a-7284-4599-b100-c2a8677c1cee None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:55.565428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856e07a-7284-4599-b100-c2a8677c1cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:55.584782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b856e07a-7284-4599-b100-c2a8677c1cee None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:55.586245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b856e07a-7284-4599-b100-c2a8677c1cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:55.595578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1063/2103] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:55.599818 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253fe3f-cf14-458c-86df-df4828bac7f6 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:55.610337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253fe3f-cf14-458c-86df-df4828bac7f6 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:55.615872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f285a76-a5f4-4c19-b9c8-4360999c3a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:55.616018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f285a76-a5f4-4c19-b9c8-4360999c3a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:55.616191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f285a76-a5f4-4c19-b9c8-4360999c3a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:55.616666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f285a76-a5f4-4c19-b9c8-4360999c3a13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:55.617488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f285a76-a5f4-4c19-b9c8-4360999c3a13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:54:55.617958 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f285a76-a5f4-4c19-b9c8-4360999c3a13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:55.619770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f285a76-a5f4-4c19-b9c8-4360999c3a13 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:55.633477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1041/2104] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:54:55.637642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856e07a-7284-4599-b100-c2a8677c1cee None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:55.648565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856e07a-7284-4599-b100-c2a8677c1cee None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:55.653363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a806b3-9606-4662-9a30-381d2c26e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:55.654091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a806b3-9606-4662-9a30-381d2c26e6aa admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:55.654378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a806b3-9606-4662-9a30-381d2c26e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:55.654685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a806b3-9606-4662-9a30-381d2c26e6aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:55.655336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a806b3-9606-4662-9a30-381d2c26e6aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:55.657527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a806b3-9606-4662-9a30-381d2c26e6aa admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:55.753698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1064/2105] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:55 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:55.757946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f285a76-a5f4-4c19-b9c8-4360999c3a13 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:55.769384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f285a76-a5f4-4c19-b9c8-4360999c3a13 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:55.835645 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302d591-5ad9-473b-90dc-24aa525b42ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:55.840867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302d591-5ad9-473b-90dc-24aa525b42ad admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:55.840867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302d591-5ad9-473b-90dc-24aa525b42ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:55.840867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302d591-5ad9-473b-90dc-24aa525b42ad admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:55.840867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302d591-5ad9-473b-90dc-24aa525b42ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:55.840867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302d591-5ad9-473b-90dc-24aa525b42ad admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:55.939786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1042/2106] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:55 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:55.943676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a806b3-9606-4662-9a30-381d2c26e6aa admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:55.962494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a806b3-9606-4662-9a30-381d2c26e6aa admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:55.978728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d413cbe-9bcf-4778-ac1f-baf529bea048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:55.978728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d413cbe-9bcf-4778-ac1f-baf529bea048 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:55.978728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d413cbe-9bcf-4778-ac1f-baf529bea048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:55.978728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d413cbe-9bcf-4778-ac1f-baf529bea048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:55.979186 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d413cbe-9bcf-4778-ac1f-baf529bea048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:55.979186 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d413cbe-9bcf-4778-ac1f-baf529bea048 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:55.999397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1065/2107] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:56.003346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302d591-5ad9-473b-90dc-24aa525b42ad admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:56.029353 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302d591-5ad9-473b-90dc-24aa525b42ad admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:56.035918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc1b5e-acb4-4c0b-8a82-e76a45c9d352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:56.042607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc1b5e-acb4-4c0b-8a82-e76a45c9d352 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:56.042607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc1b5e-acb4-4c0b-8a82-e76a45c9d352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:56.042607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc1b5e-acb4-4c0b-8a82-e76a45c9d352 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:56.042607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc1b5e-acb4-4c0b-8a82-e76a45c9d352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:56.047939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc1b5e-acb4-4c0b-8a82-e76a45c9d352 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:56.076153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1043/2108] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:56.081885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d413cbe-9bcf-4778-ac1f-baf529bea048 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:56.102671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d413cbe-9bcf-4778-ac1f-baf529bea048 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:56.119828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d85895-a09e-4349-92d8-e554c4decf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:56.120622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d85895-a09e-4349-92d8-e554c4decf98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:56.125002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d85895-a09e-4349-92d8-e554c4decf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:56.125369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d85895-a09e-4349-92d8-e554c4decf98 admin admin] PATH_INFO: `/v3/projects/2132bf8417c447f59caee55f878ea4dd/users/7ae8fe9f14054ac3bb34a711d86f35b2/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:56.148700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d85895-a09e-4349-92d8-e554c4decf98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2132bf8417c447f59caee55f878ea4dd, user_id=7ae8fe9f14054ac3bb34a711d86f35b2, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:56.151791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d85895-a09e-4349-92d8-e554c4decf98 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:56.296815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1066/2109] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:56 2022] PUT /identity/v3/projects/2132bf8417c447f59caee55f878ea4dd/users/7ae8fe9f14054ac3bb34a711d86f35b2/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:54:56.308452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307efac-6db4-4706-b8c5-5c75c572ef5d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:56.308965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307efac-6db4-4706-b8c5-5c75c572ef5d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:56.309442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307efac-6db4-4706-b8c5-5c75c572ef5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:56.358641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6307efac-6db4-4706-b8c5-5c75c572ef5d None None] MFA Rules not processed for user `7ae8fe9f14054ac3bb34a711d86f35b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:56.360032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6307efac-6db4-4706-b8c5-5c75c572ef5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:56.431750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1044/2110] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:56.436421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d85895-a09e-4349-92d8-e554c4decf98 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:56.510945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d85895-a09e-4349-92d8-e554c4decf98 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:56.517626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc0f34-1dc5-49a2-b7ef-363c7fe7dd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:56.518253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc0f34-1dc5-49a2-b7ef-363c7fe7dd1b admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:56.518568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc0f34-1dc5-49a2-b7ef-363c7fe7dd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:56.518854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc0f34-1dc5-49a2-b7ef-363c7fe7dd1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:56.519513 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc0f34-1dc5-49a2-b7ef-363c7fe7dd1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:56.521463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc0f34-1dc5-49a2-b7ef-363c7fe7dd1b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:56.664142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1067/2111] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:56 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:56.665133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307efac-6db4-4706-b8c5-5c75c572ef5d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:56.677127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307efac-6db4-4706-b8c5-5c75c572ef5d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:56.719449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:56.745761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd040ac-ad19-486b-ad65-aef975bb5501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:56.746711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd040ac-ad19-486b-ad65-aef975bb5501 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:56.747170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd040ac-ad19-486b-ad65-aef975bb5501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:56.747611 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd040ac-ad19-486b-ad65-aef975bb5501 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:56.748452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd040ac-ad19-486b-ad65-aef975bb5501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:56.751862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd040ac-ad19-486b-ad65-aef975bb5501 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:56.876360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1045/2112] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:56 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:56.880472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc0f34-1dc5-49a2-b7ef-363c7fe7dd1b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:56.891725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc0f34-1dc5-49a2-b7ef-363c7fe7dd1b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:56.896458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc050354-effe-42ef-b446-2ed932ee21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:56.897072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc050354-effe-42ef-b446-2ed932ee21dd admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:56.897352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc050354-effe-42ef-b446-2ed932ee21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:56.897626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc050354-effe-42ef-b446-2ed932ee21dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:56.898276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc050354-effe-42ef-b446-2ed932ee21dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:56.900004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc050354-effe-42ef-b446-2ed932ee21dd admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:56.912630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1068/2113] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:54:56.916972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd040ac-ad19-486b-ad65-aef975bb5501 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:56.929211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd040ac-ad19-486b-ad65-aef975bb5501 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:56.933875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea84c3-9c50-45b6-86ed-0d04823c696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:56.934557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea84c3-9c50-45b6-86ed-0d04823c696c admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:56.934846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea84c3-9c50-45b6-86ed-0d04823c696c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:56.935127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea84c3-9c50-45b6-86ed-0d04823c696c admin admin] PATH_INFO: `/v3/projects/3bd31e8091294e4e8aaf14e7f16731af/users/01e712f394a64d628f3e89410eecb2f2/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:56.953519 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea84c3-9c50-45b6-86ed-0d04823c696c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd31e8091294e4e8aaf14e7f16731af, user_id=01e712f394a64d628f3e89410eecb2f2, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:56.956271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea84c3-9c50-45b6-86ed-0d04823c696c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.066767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1046/2114] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:56 2022] PUT /identity/v3/projects/3bd31e8091294e4e8aaf14e7f16731af/users/01e712f394a64d628f3e89410eecb2f2/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:57.069956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc050354-effe-42ef-b446-2ed932ee21dd admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.085146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc050354-effe-42ef-b446-2ed932ee21dd admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.134726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:57.207708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aff73-d5bf-4237-9cbb-c1bd0bb4331c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.208092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aff73-d5bf-4237-9cbb-c1bd0bb4331c admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.208334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aff73-d5bf-4237-9cbb-c1bd0bb4331c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.208623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aff73-d5bf-4237-9cbb-c1bd0bb4331c admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.209408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aff73-d5bf-4237-9cbb-c1bd0bb4331c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.211079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aff73-d5bf-4237-9cbb-c1bd0bb4331c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.223891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1069/2115] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:57.231689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea84c3-9c50-45b6-86ed-0d04823c696c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.245511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea84c3-9c50-45b6-86ed-0d04823c696c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.250496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc224f7-406b-421c-8459-966c4ad9aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.251217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc224f7-406b-421c-8459-966c4ad9aa22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.251502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc224f7-406b-421c-8459-966c4ad9aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.251784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc224f7-406b-421c-8459-966c4ad9aa22 admin admin] PATH_INFO: `/v3/projects/3bd31e8091294e4e8aaf14e7f16731af/users/01e712f394a64d628f3e89410eecb2f2/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.271501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc224f7-406b-421c-8459-966c4ad9aa22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd31e8091294e4e8aaf14e7f16731af, user_id=01e712f394a64d628f3e89410eecb2f2, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.274229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc224f7-406b-421c-8459-966c4ad9aa22 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.299531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1047/2116] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:57 2022] PUT /identity/v3/projects/3bd31e8091294e4e8aaf14e7f16731af/users/01e712f394a64d628f3e89410eecb2f2/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:57.309765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561f85b-739e-4ddb-a164-530019fa9b15 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.310415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561f85b-739e-4ddb-a164-530019fa9b15 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.311066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561f85b-739e-4ddb-a164-530019fa9b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.345005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-0561f85b-739e-4ddb-a164-530019fa9b15 None None] MFA Rules not processed for user `01e712f394a64d628f3e89410eecb2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:57.346253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-0561f85b-739e-4ddb-a164-530019fa9b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:57.454222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1070/2117] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:57.466160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc224f7-406b-421c-8459-966c4ad9aa22 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.468574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8dc224f7-406b-421c-8459-966c4ad9aa22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:57.572799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc224f7-406b-421c-8459-966c4ad9aa22 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.577868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8c551-6561-47fb-9dbd-6856dfcf4fe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.579954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8c551-6561-47fb-9dbd-6856dfcf4fe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.580275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8c551-6561-47fb-9dbd-6856dfcf4fe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.580869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8c551-6561-47fb-9dbd-6856dfcf4fe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.602592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8c551-6561-47fb-9dbd-6856dfcf4fe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.604464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8c551-6561-47fb-9dbd-6856dfcf4fe9 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.617203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1048/2118] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:57.708264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561f85b-739e-4ddb-a164-530019fa9b15 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.794707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561f85b-739e-4ddb-a164-530019fa9b15 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.799657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.800316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.800601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.800882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] PATH_INFO: `/v3/users/7ae8fe9f14054ac3bb34a711d86f35b2` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.802031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae8fe9f14054ac3bb34a711d86f35b2)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.804084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:58.659090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae8fe9f14054ac3bb34a711d86f35b2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:58.659803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae8fe9f14054ac3bb34a711d86f35b2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:58.800771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae8fe9f14054ac3bb34a711d86f35b2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:58.812936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae8fe9f14054ac3bb34a711d86f35b2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:58.815816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:58.827678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae8fe9f14054ac3bb34a711d86f35b2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:58.837352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1071/2119] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:57 2022] DELETE /identity/v3/users/7ae8fe9f14054ac3bb34a711d86f35b2 => generated 0 bytes in 1130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:58.841967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76521d3-95bf-4889-87d0-410a5eba0443 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:58.843092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76521d3-95bf-4889-87d0-410a5eba0443 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:58.843092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76521d3-95bf-4889-87d0-410a5eba0443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:58.864777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f76521d3-95bf-4889-87d0-410a5eba0443 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:58.865739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f76521d3-95bf-4889-87d0-410a5eba0443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:58.947533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1049/2120] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:58.958996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:58.963836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:59.029007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e538b2f-6745-49ec-8a14-f0f7cc7d71a9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:59.032881 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619eb3a-a7d6-4d6d-b104-dd14a8074522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:59.033559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619eb3a-a7d6-4d6d-b104-dd14a8074522 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:59.037575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619eb3a-a7d6-4d6d-b104-dd14a8074522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:59.037575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619eb3a-a7d6-4d6d-b104-dd14a8074522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:59.043239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619eb3a-a7d6-4d6d-b104-dd14a8074522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:59.044678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619eb3a-a7d6-4d6d-b104-dd14a8074522 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:59.054736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1072/2121] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:59.651190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76521d3-95bf-4889-87d0-410a5eba0443 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:59.652951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f76521d3-95bf-4889-87d0-410a5eba0443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:59.670124 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76521d3-95bf-4889-87d0-410a5eba0443 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:59.682465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:59.686360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:59.686688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:59.687705 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] PATH_INFO: `/v3/projects/2132bf8417c447f59caee55f878ea4dd` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:59.689603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2132bf8417c447f59caee55f878ea4dd)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:59.691988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:59.833465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:59.864567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2132bf8417c447f59caee55f878ea4dd', 'request_id': 'req-24974bf2-8161-4d02-ab4a-e4c6e55297c5'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:59.967859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2132bf8417c447f59caee55f878ea4dd', 'request_id': 'req-24974bf2-8161-4d02-ab4a-e4c6e55297c5'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:59.987122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2132bf8417c447f59caee55f878ea4dd', 'request_id': 'req-24974bf2-8161-4d02-ab4a-e4c6e55297c5'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:59.995236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] The token cache is being invalidate because project 2132bf8417c447f59caee55f878ea4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:59.995966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:59.998146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:00.001760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1050/2122] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:59 2022] DELETE /identity/v3/projects/2132bf8417c447f59caee55f878ea4dd => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:00.007767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619eb3a-a7d6-4d6d-b104-dd14a8074522 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:00.019838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619eb3a-a7d6-4d6d-b104-dd14a8074522 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:00.120150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:00.120865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:00.121153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:00.121485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] PATH_INFO: `/v3/users/01e712f394a64d628f3e89410eecb2f2` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:00.123888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e712f394a64d628f3e89410eecb2f2)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:00.127614 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:00.322473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e712f394a64d628f3e89410eecb2f2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:00.323157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e712f394a64d628f3e89410eecb2f2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:00.389243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e712f394a64d628f3e89410eecb2f2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:00.397225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e712f394a64d628f3e89410eecb2f2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:00.410605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a01d22d0-bdb9-4393-86e2-de7ca4d05039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e712f394a64d628f3e89410eecb2f2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:00.416821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1073/2123] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:55:00 2022] DELETE /identity/v3/users/01e712f394a64d628f3e89410eecb2f2 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:00.911206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:00.912919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:00.926456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24974bf2-8161-4d02-ab4a-e4c6e55297c5 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:00.991213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:00.991965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:00.992257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:00.992533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] PATH_INFO: `/v3/projects/3bd31e8091294e4e8aaf14e7f16731af` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:00.993917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd31e8091294e4e8aaf14e7f16731af)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:00.996243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:01.145072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd31e8091294e4e8aaf14e7f16731af', 'request_id': 'req-a02620a6-094f-41ea-b500-d20577d0dbef'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:01.247035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd31e8091294e4e8aaf14e7f16731af', 'request_id': 'req-a02620a6-094f-41ea-b500-d20577d0dbef'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:01.255393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd31e8091294e4e8aaf14e7f16731af', 'request_id': 'req-a02620a6-094f-41ea-b500-d20577d0dbef'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:01.269098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] The token cache is being invalidate because project 3bd31e8091294e4e8aaf14e7f16731af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:55:01.269424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:01.270086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a02620a6-094f-41ea-b500-d20577d0dbef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:01.271835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1051/2124] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:55:00 2022] DELETE /identity/v3/projects/3bd31e8091294e4e8aaf14e7f16731af => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:01.276433 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c446ca-2f85-4e2a-8ced-d0b744469028 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:01.276939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c446ca-2f85-4e2a-8ced-d0b744469028 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:01.277050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c446ca-2f85-4e2a-8ced-d0b744469028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:01.294471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:01.309768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-04c446ca-2f85-4e2a-8ced-d0b744469028 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:01.311171 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-04c446ca-2f85-4e2a-8ced-d0b744469028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:01.397514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1074/2125] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:01.404927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfd059-1c31-4704-a9f6-180f1377e84a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:01.405218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfd059-1c31-4704-a9f6-180f1377e84a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:01.405541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfd059-1c31-4704-a9f6-180f1377e84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:01.460565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-36bfd059-1c31-4704-a9f6-180f1377e84a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:01.460565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-36bfd059-1c31-4704-a9f6-180f1377e84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:01.461698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c446ca-2f85-4e2a-8ced-d0b744469028 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:01.462370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1052/2126] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:01.488086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c446ca-2f85-4e2a-8ced-d0b744469028 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:01.494447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f5c8f-ea6f-4b78-bd0f-8162abdbf06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:01.495541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f5c8f-ea6f-4b78-bd0f-8162abdbf06b admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:01.496940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f5c8f-ea6f-4b78-bd0f-8162abdbf06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:01.497412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f5c8f-ea6f-4b78-bd0f-8162abdbf06b admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:01.498876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f5c8f-ea6f-4b78-bd0f-8162abdbf06b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:55:01.500254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f5c8f-ea6f-4b78-bd0f-8162abdbf06b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:01.502406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f5c8f-ea6f-4b78-bd0f-8162abdbf06b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:01.517125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1075/2127] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:55:01.521536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfd059-1c31-4704-a9f6-180f1377e84a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:01.533102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfd059-1c31-4704-a9f6-180f1377e84a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:01.538438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880a57-f638-4b8d-b5b2-1582e4de1287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:01.539116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880a57-f638-4b8d-b5b2-1582e4de1287 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:01.539400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880a57-f638-4b8d-b5b2-1582e4de1287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:01.539673 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880a57-f638-4b8d-b5b2-1582e4de1287 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:01.540337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d880a57-f638-4b8d-b5b2-1582e4de1287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:01.542286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d880a57-f638-4b8d-b5b2-1582e4de1287 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:01.723205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1053/2128] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:55:01 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:01.727460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f5c8f-ea6f-4b78-bd0f-8162abdbf06b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:01.738588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f5c8f-ea6f-4b78-bd0f-8162abdbf06b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:01.797964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01979d70-21dd-49dd-ae66-e6a02b3a0567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:01.798602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01979d70-21dd-49dd-ae66-e6a02b3a0567 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:01.798947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01979d70-21dd-49dd-ae66-e6a02b3a0567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:01.799264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01979d70-21dd-49dd-ae66-e6a02b3a0567 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:01.799911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01979d70-21dd-49dd-ae66-e6a02b3a0567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:01.801861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01979d70-21dd-49dd-ae66-e6a02b3a0567 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:01.920926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1076/2129] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:55:01 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:01.925626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880a57-f638-4b8d-b5b2-1582e4de1287 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:01.936316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880a57-f638-4b8d-b5b2-1582e4de1287 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:01.941791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc6587-d98d-4172-a9dc-290bd2bad4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:01.942414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc6587-d98d-4172-a9dc-290bd2bad4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:01.942739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc6587-d98d-4172-a9dc-290bd2bad4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:01.943021 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc6587-d98d-4172-a9dc-290bd2bad4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:01.943683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc6587-d98d-4172-a9dc-290bd2bad4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:01.945306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc6587-d98d-4172-a9dc-290bd2bad4f8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:01.956207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1054/2130] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:01.960234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01979d70-21dd-49dd-ae66-e6a02b3a0567 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:01.971444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01979d70-21dd-49dd-ae66-e6a02b3a0567 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:01.977883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8adc2-c405-428d-bf24-6eda9c6996db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:01.978640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8adc2-c405-428d-bf24-6eda9c6996db admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:01.979015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8adc2-c405-428d-bf24-6eda9c6996db admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:01.979363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8adc2-c405-428d-bf24-6eda9c6996db admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:01.980228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8adc2-c405-428d-bf24-6eda9c6996db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:01.982438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb8adc2-c405-428d-bf24-6eda9c6996db admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:01.995219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1077/2131] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:55:01.999247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc6587-d98d-4172-a9dc-290bd2bad4f8 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:02.010574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc6587-d98d-4172-a9dc-290bd2bad4f8 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:02.019253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262f7eb-7f54-4b12-8720-14ffdc1178e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:02.020192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262f7eb-7f54-4b12-8720-14ffdc1178e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:02.020479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262f7eb-7f54-4b12-8720-14ffdc1178e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:02.020814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262f7eb-7f54-4b12-8720-14ffdc1178e2 admin admin] PATH_INFO: `/v3/projects/d5817def63484eafba13cf7639bf40ed/users/b9379ff5260a497b924298e77fc772b9/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:02.028823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:02.041612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262f7eb-7f54-4b12-8720-14ffdc1178e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5817def63484eafba13cf7639bf40ed, user_id=b9379ff5260a497b924298e77fc772b9, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:02.044429 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262f7eb-7f54-4b12-8720-14ffdc1178e2 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:02.138635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1055/2132] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:01 2022] PUT /identity/v3/projects/d5817def63484eafba13cf7639bf40ed/users/b9379ff5260a497b924298e77fc772b9/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:02.151510 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ecb01-43bd-43be-a5ae-4f9cdbd254d8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:02.151810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ecb01-43bd-43be-a5ae-4f9cdbd254d8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:02.152089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ecb01-43bd-43be-a5ae-4f9cdbd254d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:02.180402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-141ecb01-43bd-43be-a5ae-4f9cdbd254d8 None None] MFA Rules not processed for user `b9379ff5260a497b924298e77fc772b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:02.181674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-141ecb01-43bd-43be-a5ae-4f9cdbd254d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:02.247288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:02.251263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1078/2133] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:02.255793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262f7eb-7f54-4b12-8720-14ffdc1178e2 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:02.320746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262f7eb-7f54-4b12-8720-14ffdc1178e2 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:02.325462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545c49e-069e-4ce1-9d4d-b875374d319a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:02.326070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545c49e-069e-4ce1-9d4d-b875374d319a admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:02.326362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545c49e-069e-4ce1-9d4d-b875374d319a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:02.326665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545c49e-069e-4ce1-9d4d-b875374d319a admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:02.327977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545c49e-069e-4ce1-9d4d-b875374d319a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:02.329991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545c49e-069e-4ce1-9d4d-b875374d319a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:02.425309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1056/2134] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:55:02 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:02.429371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ecb01-43bd-43be-a5ae-4f9cdbd254d8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:02.440773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ecb01-43bd-43be-a5ae-4f9cdbd254d8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:02.501114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d30ef2-20c6-48db-b2c3-947037aee360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:02.501883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d30ef2-20c6-48db-b2c3-947037aee360 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:02.502232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d30ef2-20c6-48db-b2c3-947037aee360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:02.502643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d30ef2-20c6-48db-b2c3-947037aee360 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:02.503341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d30ef2-20c6-48db-b2c3-947037aee360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:02.505741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d30ef2-20c6-48db-b2c3-947037aee360 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:02.601892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1079/2135] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:55:02 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:02.606184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545c49e-069e-4ce1-9d4d-b875374d319a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:02.618342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545c49e-069e-4ce1-9d4d-b875374d319a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:02.623499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034b3d5-e676-4f93-8b47-2369bf0ebe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:02.625783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034b3d5-e676-4f93-8b47-2369bf0ebe2c admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:02.625783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034b3d5-e676-4f93-8b47-2369bf0ebe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:02.625783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034b3d5-e676-4f93-8b47-2369bf0ebe2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:02.626441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034b3d5-e676-4f93-8b47-2369bf0ebe2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:02.628212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034b3d5-e676-4f93-8b47-2369bf0ebe2c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:02.643628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1057/2136] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:02.645613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d30ef2-20c6-48db-b2c3-947037aee360 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:02.656773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d30ef2-20c6-48db-b2c3-947037aee360 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:02.667953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c7cf9-1f59-4cb0-8982-a42d21da201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:02.668577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c7cf9-1f59-4cb0-8982-a42d21da201b admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:02.668858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c7cf9-1f59-4cb0-8982-a42d21da201b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:02.669130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c7cf9-1f59-4cb0-8982-a42d21da201b admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:02.669786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c7cf9-1f59-4cb0-8982-a42d21da201b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:02.671815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c7cf9-1f59-4cb0-8982-a42d21da201b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:02.682846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1080/2137] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:02.687629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034b3d5-e676-4f93-8b47-2369bf0ebe2c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:02.697771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034b3d5-e676-4f93-8b47-2369bf0ebe2c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:02.703216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a380ab-a656-4c55-9075-53729cc09774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:02.703867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a380ab-a656-4c55-9075-53729cc09774 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:02.704142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a380ab-a656-4c55-9075-53729cc09774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:02.704434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a380ab-a656-4c55-9075-53729cc09774 admin admin] PATH_INFO: `/v3/projects/631a2038f46b4b328e6504cfdfb8b8a1/users/0aaa49a2d0524f44ba007b5664918ab3/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:02.721872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a380ab-a656-4c55-9075-53729cc09774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=631a2038f46b4b328e6504cfdfb8b8a1, user_id=0aaa49a2d0524f44ba007b5664918ab3, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:02.725038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a380ab-a656-4c55-9075-53729cc09774 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:02.841138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1058/2138] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:02 2022] PUT /identity/v3/projects/631a2038f46b4b328e6504cfdfb8b8a1/users/0aaa49a2d0524f44ba007b5664918ab3/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:02.851843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac0f1f-8908-4b82-a5a2-fb4e14c2f6a7 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:02.852134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac0f1f-8908-4b82-a5a2-fb4e14c2f6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:02.852416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac0f1f-8908-4b82-a5a2-fb4e14c2f6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:02.885054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d2ac0f1f-8908-4b82-a5a2-fb4e14c2f6a7 None None] MFA Rules not processed for user `0aaa49a2d0524f44ba007b5664918ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:02.889101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d2ac0f1f-8908-4b82-a5a2-fb4e14c2f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:02.958181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1081/2139] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:02.962972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a380ab-a656-4c55-9075-53729cc09774 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:03.026666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a380ab-a656-4c55-9075-53729cc09774 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:03.031469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03612a9-2c1b-4c86-bf89-dd5d6cf85d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:03.032074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03612a9-2c1b-4c86-bf89-dd5d6cf85d35 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:03.032355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03612a9-2c1b-4c86-bf89-dd5d6cf85d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:03.032628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03612a9-2c1b-4c86-bf89-dd5d6cf85d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:03.033302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03612a9-2c1b-4c86-bf89-dd5d6cf85d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:03.035282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03612a9-2c1b-4c86-bf89-dd5d6cf85d35 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:03.164399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1059/2140] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:55:02 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:03.168016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac0f1f-8908-4b82-a5a2-fb4e14c2f6a7 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:03.184282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac0f1f-8908-4b82-a5a2-fb4e14c2f6a7 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:03.271643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab7c7e-c08b-45e0-b9fc-21518b08e895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:03.272294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab7c7e-c08b-45e0-b9fc-21518b08e895 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:03.272582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab7c7e-c08b-45e0-b9fc-21518b08e895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:03.272911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab7c7e-c08b-45e0-b9fc-21518b08e895 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:03.273565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab7c7e-c08b-45e0-b9fc-21518b08e895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:03.275609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab7c7e-c08b-45e0-b9fc-21518b08e895 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:03.363046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:03.379080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1082/2141] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:55:03 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:03.379738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03612a9-2c1b-4c86-bf89-dd5d6cf85d35 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:03.394558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03612a9-2c1b-4c86-bf89-dd5d6cf85d35 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:03.422650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd96d7-68da-45f3-b76d-bee30d573660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:03.422650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd96d7-68da-45f3-b76d-bee30d573660 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:03.422650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd96d7-68da-45f3-b76d-bee30d573660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:03.422650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd96d7-68da-45f3-b76d-bee30d573660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:03.423320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd96d7-68da-45f3-b76d-bee30d573660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:03.442718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd96d7-68da-45f3-b76d-bee30d573660 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:03.482293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1060/2142] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:03.484206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab7c7e-c08b-45e0-b9fc-21518b08e895 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:03.498789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab7c7e-c08b-45e0-b9fc-21518b08e895 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:03.515953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae483f-4199-458a-af9e-c8399aa4b0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:03.516980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae483f-4199-458a-af9e-c8399aa4b0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:03.517464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae483f-4199-458a-af9e-c8399aa4b0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:03.517925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae483f-4199-458a-af9e-c8399aa4b0f5 admin admin] PATH_INFO: `/v3/projects/09423249eeb0495a8e6c0dbff8b6798b/users/914521844fc6441c87bc6325a5f186c5/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:03.543259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae483f-4199-458a-af9e-c8399aa4b0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09423249eeb0495a8e6c0dbff8b6798b, user_id=914521844fc6441c87bc6325a5f186c5, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:03.546992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae483f-4199-458a-af9e-c8399aa4b0f5 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:03.699745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1083/2143] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:03 2022] PUT /identity/v3/projects/09423249eeb0495a8e6c0dbff8b6798b/users/914521844fc6441c87bc6325a5f186c5/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:03.704193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd96d7-68da-45f3-b76d-bee30d573660 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:03.723363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd96d7-68da-45f3-b76d-bee30d573660 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:03.733606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:03.812898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a883f6-bb54-437e-bdef-342625d13add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:03.813905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a883f6-bb54-437e-bdef-342625d13add admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:03.814345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a883f6-bb54-437e-bdef-342625d13add admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:03.814809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a883f6-bb54-437e-bdef-342625d13add admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:03.815878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a883f6-bb54-437e-bdef-342625d13add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:03.817714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a883f6-bb54-437e-bdef-342625d13add admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:03.831913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1061/2144] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:55:03.836166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae483f-4199-458a-af9e-c8399aa4b0f5 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:03.852933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae483f-4199-458a-af9e-c8399aa4b0f5 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:03.865815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b21082-81b4-4e32-8c51-0d42d6bbf49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:03.866391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b21082-81b4-4e32-8c51-0d42d6bbf49b admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:03.866755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b21082-81b4-4e32-8c51-0d42d6bbf49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:03.867386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b21082-81b4-4e32-8c51-0d42d6bbf49b admin admin] PATH_INFO: `/v3/projects/09423249eeb0495a8e6c0dbff8b6798b/users/914521844fc6441c87bc6325a5f186c5/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:03.870251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b21082-81b4-4e32-8c51-0d42d6bbf49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=09423249eeb0495a8e6c0dbff8b6798b, user_id=914521844fc6441c87bc6325a5f186c5, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:03.873479 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b21082-81b4-4e32-8c51-0d42d6bbf49b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:03.885980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1084/2145] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:03 2022] PUT /identity/v3/projects/09423249eeb0495a8e6c0dbff8b6798b/users/914521844fc6441c87bc6325a5f186c5/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:03.899554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d750b1-986d-4bba-870a-3ff477548ee7 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:03.900804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d750b1-986d-4bba-870a-3ff477548ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:03.900804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d750b1-986d-4bba-870a-3ff477548ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:03.936885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-24d750b1-986d-4bba-870a-3ff477548ee7 None None] MFA Rules not processed for user `914521844fc6441c87bc6325a5f186c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:03.938485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-24d750b1-986d-4bba-870a-3ff477548ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:04.059796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1062/2146] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:04.068016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b21082-81b4-4e32-8c51-0d42d6bbf49b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:04.070088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-07b21082-81b4-4e32-8c51-0d42d6bbf49b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:04.139305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b21082-81b4-4e32-8c51-0d42d6bbf49b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:04.147240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df1a56-151f-4b30-bc3e-d2d36be6f602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:04.148065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df1a56-151f-4b30-bc3e-d2d36be6f602 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:04.148361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df1a56-151f-4b30-bc3e-d2d36be6f602 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:04.148643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df1a56-151f-4b30-bc3e-d2d36be6f602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:04.162249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df1a56-151f-4b30-bc3e-d2d36be6f602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:04.163833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df1a56-151f-4b30-bc3e-d2d36be6f602 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:04.186006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1085/2147] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:04.331977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d750b1-986d-4bba-870a-3ff477548ee7 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:04.344779 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d750b1-986d-4bba-870a-3ff477548ee7 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:04.349283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4b268-eaf4-4f84-a40c-b234b910d1c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:04.349903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4b268-eaf4-4f84-a40c-b234b910d1c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:04.350190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4b268-eaf4-4f84-a40c-b234b910d1c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:04.350470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4b268-eaf4-4f84-a40c-b234b910d1c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:04.416529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4b268-eaf4-4f84-a40c-b234b910d1c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:04.421126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4b268-eaf4-4f84-a40c-b234b910d1c1 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:04.453903 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1063/2148] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:06.706678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39034c4e-244d-4d5d-bc2e-f5ebbc79f001 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:06.707259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39034c4e-244d-4d5d-bc2e-f5ebbc79f001 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:06.707650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39034c4e-244d-4d5d-bc2e-f5ebbc79f001 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:06.709380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1086/2149] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:55:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:55:06.716088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98537e8-ac47-471c-9a51-3394b7c215a8 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:06.716605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98537e8-ac47-471c-9a51-3394b7c215a8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:06.716889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98537e8-ac47-471c-9a51-3394b7c215a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:06.741649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c98537e8-ac47-471c-9a51-3394b7c215a8 None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:06.744229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c98537e8-ac47-471c-9a51-3394b7c215a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:06.772473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:06.836456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1064/2150] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:55:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:06.850170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39034c4e-244d-4d5d-bc2e-f5ebbc79f001 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:06.852271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-39034c4e-244d-4d5d-bc2e-f5ebbc79f001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:06.924576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39034c4e-244d-4d5d-bc2e-f5ebbc79f001 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:06.930214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5d5a8-a3ab-46af-ae59-a5e84f72c048 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ff3f53ddbb04eb8916b9ccfdb03dd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:06.931097 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5d5a8-a3ab-46af-ae59-a5e84f72c048 service glance] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:06.931448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5d5a8-a3ab-46af-ae59-a5e84f72c048 service glance] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:06.932408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5d5a8-a3ab-46af-ae59-a5e84f72c048 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:06.945778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5d5a8-a3ab-46af-ae59-a5e84f72c048 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:06.947656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5d5a8-a3ab-46af-ae59-a5e84f72c048 service glance] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:06.962083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1087/2151] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:07.195079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53add7a4-753e-4c13-9923-37ad5d5593e7 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:07.195784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53add7a4-753e-4c13-9923-37ad5d5593e7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:07.196263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53add7a4-753e-4c13-9923-37ad5d5593e7 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:07.198947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1065/2152] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:55:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:55:07.206369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a1c80-9a29-4feb-a078-d17e69642023 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:07.206793 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a1c80-9a29-4feb-a078-d17e69642023 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:07.207075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a1c80-9a29-4feb-a078-d17e69642023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:07.238259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d78a1c80-9a29-4feb-a078-d17e69642023 None None] MFA Rules not processed for user `f81cc77e85f84d358471dd9efab45cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:07.245409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d78a1c80-9a29-4feb-a078-d17e69642023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:07.300372 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1088/2153] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:55:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:07.316252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53add7a4-753e-4c13-9923-37ad5d5593e7 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:07.318291 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-53add7a4-753e-4c13-9923-37ad5d5593e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:07.337033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53add7a4-753e-4c13-9923-37ad5d5593e7 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:07.340921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407edb72-1a2e-4442-a68a-90964da6bad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:07.341553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407edb72-1a2e-4442-a68a-90964da6bad4 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:07.341839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407edb72-1a2e-4442-a68a-90964da6bad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:07.342174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407edb72-1a2e-4442-a68a-90964da6bad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:07.350883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407edb72-1a2e-4442-a68a-90964da6bad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:07.351888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407edb72-1a2e-4442-a68a-90964da6bad4 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:07.362287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1066/2154] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:55:07.504540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5fffc-570a-491a-91a3-1a88810ea497 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:07.505703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5fffc-570a-491a-91a3-1a88810ea497 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:07.506436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5fffc-570a-491a-91a3-1a88810ea497 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:07.510145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1089/2155] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:55:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:55:07.518602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930088b5-e2ac-47c3-8539-b7446068fb83 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:07.518938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930088b5-e2ac-47c3-8539-b7446068fb83 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:07.519158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930088b5-e2ac-47c3-8539-b7446068fb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:07.582078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-930088b5-e2ac-47c3-8539-b7446068fb83 None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:07.583664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-930088b5-e2ac-47c3-8539-b7446068fb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:07.604846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1067/2156] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:55:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:55:07.616541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5fffc-570a-491a-91a3-1a88810ea497 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:07.623241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-00f5fffc-570a-491a-91a3-1a88810ea497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:07.641068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5fffc-570a-491a-91a3-1a88810ea497 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:07.644753 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b358c7-761e-4b3d-95b1-6077943b8739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:07.645778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b358c7-761e-4b3d-95b1-6077943b8739 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:07.645778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b358c7-761e-4b3d-95b1-6077943b8739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:07.646002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b358c7-761e-4b3d-95b1-6077943b8739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:07.671540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b358c7-761e-4b3d-95b1-6077943b8739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:07.673414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b358c7-761e-4b3d-95b1-6077943b8739 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:07.712158 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1090/2157] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:55:09.797826 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930088b5-e2ac-47c3-8539-b7446068fb83 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:09.883334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930088b5-e2ac-47c3-8539-b7446068fb83 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:09.888199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:09.889966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:09.890249 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:09.890727 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] PATH_INFO: `/v3/users/b9379ff5260a497b924298e77fc772b9` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:09.891927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9379ff5260a497b924298e77fc772b9)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:09.893926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:10.110913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9379ff5260a497b924298e77fc772b9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:10.153703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9379ff5260a497b924298e77fc772b9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:10.166382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9379ff5260a497b924298e77fc772b9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:10.173957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9379ff5260a497b924298e77fc772b9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:10.174670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9379ff5260a497b924298e77fc772b9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:10.180524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1068/2158] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:55:09 2022] DELETE /identity/v3/users/b9379ff5260a497b924298e77fc772b9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:10.185165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a20836-07a9-4afd-886b-1404b49815c1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:10.186153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a20836-07a9-4afd-886b-1404b49815c1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:10.186258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a20836-07a9-4afd-886b-1404b49815c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:10.193528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:10.207004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-08a20836-07a9-4afd-886b-1404b49815c1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:10.208409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-08a20836-07a9-4afd-886b-1404b49815c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:10.290855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1091/2159] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:10.304009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:10.304876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:10.337218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:10.403549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1b2cb-e36f-4118-aaa7-1c799e154e8c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:10.408861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56fda1-ead4-4f35-b6ef-915e6d7357cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:10.409527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56fda1-ead4-4f35-b6ef-915e6d7357cf service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:10.409811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56fda1-ead4-4f35-b6ef-915e6d7357cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:10.410085 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56fda1-ead4-4f35-b6ef-915e6d7357cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:10.422662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56fda1-ead4-4f35-b6ef-915e6d7357cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:10.424788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56fda1-ead4-4f35-b6ef-915e6d7357cf service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:10.452333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1069/2160] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:11.029618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a20836-07a9-4afd-886b-1404b49815c1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:11.031842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-08a20836-07a9-4afd-886b-1404b49815c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:11.044406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a20836-07a9-4afd-886b-1404b49815c1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:11.051654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:11.052332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:11.052621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:11.052902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] PATH_INFO: `/v3/projects/d5817def63484eafba13cf7639bf40ed` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:11.054095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5817def63484eafba13cf7639bf40ed)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:11.056116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:11.183298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5817def63484eafba13cf7639bf40ed', 'request_id': 'req-bb28cf76-42d8-49f3-b1a2-4070f891f228'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.189244 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5817def63484eafba13cf7639bf40ed', 'request_id': 'req-bb28cf76-42d8-49f3-b1a2-4070f891f228'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.342732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5817def63484eafba13cf7639bf40ed', 'request_id': 'req-bb28cf76-42d8-49f3-b1a2-4070f891f228'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.355005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] The token cache is being invalidate because project d5817def63484eafba13cf7639bf40ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:55:11.355312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.356095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.359080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1092/2161] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:55:11 2022] DELETE /identity/v3/projects/d5817def63484eafba13cf7639bf40ed => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:11.365655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56fda1-ead4-4f35-b6ef-915e6d7357cf service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:11.377433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56fda1-ead4-4f35-b6ef-915e6d7357cf service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:11.450532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:11.451886 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:11.452203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:11.452548 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] PATH_INFO: `/v3/users/0aaa49a2d0524f44ba007b5664918ab3` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:11.454500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aaa49a2d0524f44ba007b5664918ab3)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:11.457059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:11.720546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aaa49a2d0524f44ba007b5664918ab3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.874706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aaa49a2d0524f44ba007b5664918ab3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.888069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aaa49a2d0524f44ba007b5664918ab3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.895837 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aaa49a2d0524f44ba007b5664918ab3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.896567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aaa49a2d0524f44ba007b5664918ab3'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:11.902614 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1070/2162] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:55:11 2022] DELETE /identity/v3/users/0aaa49a2d0524f44ba007b5664918ab3 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:12.791719 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:12.800267 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:12.817770 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28cf76-42d8-49f3-b1a2-4070f891f228 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:12.892826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:12.893577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:12.894152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:12.894493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] PATH_INFO: `/v3/projects/631a2038f46b4b328e6504cfdfb8b8a1` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:12.913223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=631a2038f46b4b328e6504cfdfb8b8a1)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:12.913223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:13.147995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:13.163561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631a2038f46b4b328e6504cfdfb8b8a1', 'request_id': 'req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:13.170475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631a2038f46b4b328e6504cfdfb8b8a1', 'request_id': 'req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:13.308183 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631a2038f46b4b328e6504cfdfb8b8a1', 'request_id': 'req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:13.323728 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:13.325384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:13.332791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] The token cache is being invalidate because project 631a2038f46b4b328e6504cfdfb8b8a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:55:13.333283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:13.333983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:13.340100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1093/2163] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:55:12 2022] DELETE /identity/v3/projects/631a2038f46b4b328e6504cfdfb8b8a1 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:13.344495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:13.347373 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:13.408986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:13.449970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e372f80-9ee0-4aee-9dbd-64f9c9f7fec8 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:13.463233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c7d0c-ce5d-4f7e-be14-df3ca234c711 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:13.469077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:13.469729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:13.470157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:13.471601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] PATH_INFO: `/v3/users/914521844fc6441c87bc6325a5f186c5` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:13.471798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914521844fc6441c87bc6325a5f186c5)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:13.473842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:13.566423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb4a58-b991-4b3b-98ec-cbf6c3cf62c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:13.567364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb4a58-b991-4b3b-98ec-cbf6c3cf62c5 service placement] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:13.567636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb4a58-b991-4b3b-98ec-cbf6c3cf62c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:13.567998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb4a58-b991-4b3b-98ec-cbf6c3cf62c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:13.569878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-22bb4a58-b991-4b3b-98ec-cbf6c3cf62c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:13.583651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb4a58-b991-4b3b-98ec-cbf6c3cf62c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:13.585463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb4a58-b991-4b3b-98ec-cbf6c3cf62c5 service placement] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:13.602942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1071/2164] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:14.086490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914521844fc6441c87bc6325a5f186c5'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:14.094227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914521844fc6441c87bc6325a5f186c5'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:14.199008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914521844fc6441c87bc6325a5f186c5'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:14.232891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914521844fc6441c87bc6325a5f186c5'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:14.243875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4f3a9871-b413-489a-ac1c-b75ade334b94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914521844fc6441c87bc6325a5f186c5'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:14.257488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1094/2165] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:55:13 2022] DELETE /identity/v3/users/914521844fc6441c87bc6325a5f186c5 => generated 0 bytes in 909 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:14.978491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb4a58-b991-4b3b-98ec-cbf6c3cf62c5 service placement] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:14.995381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb4a58-b991-4b3b-98ec-cbf6c3cf62c5 service placement] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:15.095936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:15.097025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:15.097540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:15.097942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] PATH_INFO: `/v3/projects/09423249eeb0495a8e6c0dbff8b6798b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:15.099184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] RBAC: Authorizing `identity:delete_project(project_id=09423249eeb0495a8e6c0dbff8b6798b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:15.101201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:15.268662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09423249eeb0495a8e6c0dbff8b6798b', 'request_id': 'req-476d3f59-b8b6-4c50-8fc3-2b349690556a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:15.455751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09423249eeb0495a8e6c0dbff8b6798b', 'request_id': 'req-476d3f59-b8b6-4c50-8fc3-2b349690556a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:15.462778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09423249eeb0495a8e6c0dbff8b6798b', 'request_id': 'req-476d3f59-b8b6-4c50-8fc3-2b349690556a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:15.474832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] The token cache is being invalidate because project 09423249eeb0495a8e6c0dbff8b6798b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:55:15.475150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:15.475811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-476d3f59-b8b6-4c50-8fc3-2b349690556a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:15.477349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1072/2166] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:55:14 2022] DELETE /identity/v3/projects/09423249eeb0495a8e6c0dbff8b6798b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:15.485122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7908d-b189-4039-83e6-9e5ad80935c3 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:15.485708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7908d-b189-4039-83e6-9e5ad80935c3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:15.486629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7908d-b189-4039-83e6-9e5ad80935c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:15.510984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-77e7908d-b189-4039-83e6-9e5ad80935c3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:15.512311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-77e7908d-b189-4039-83e6-9e5ad80935c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:15.563151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:15.605042 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1095/2167] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:15.614066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463ea27-3765-43ce-a3bd-14fa60024233 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:15.614358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463ea27-3765-43ce-a3bd-14fa60024233 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:15.614762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463ea27-3765-43ce-a3bd-14fa60024233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:15.638451 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-1463ea27-3765-43ce-a3bd-14fa60024233 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:15.640292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1463ea27-3765-43ce-a3bd-14fa60024233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:15.651160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1073/2168] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:15.656694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7908d-b189-4039-83e6-9e5ad80935c3 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:15.676792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7908d-b189-4039-83e6-9e5ad80935c3 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:15.688008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad0245-67a4-4f2b-bb14-f697db3e95c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:15.689264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad0245-67a4-4f2b-bb14-f697db3e95c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:15.689898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad0245-67a4-4f2b-bb14-f697db3e95c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:15.690538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad0245-67a4-4f2b-bb14-f697db3e95c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:15.691834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad0245-67a4-4f2b-bb14-f697db3e95c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:55:15.692581 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad0245-67a4-4f2b-bb14-f697db3e95c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:15.695883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad0245-67a4-4f2b-bb14-f697db3e95c9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:15.718251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1096/2169] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:55:15.734130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463ea27-3765-43ce-a3bd-14fa60024233 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:15.746929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463ea27-3765-43ce-a3bd-14fa60024233 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:15.754857 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade9fb-d2a8-4523-97ec-aaf3dce616d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:15.755502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade9fb-d2a8-4523-97ec-aaf3dce616d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:15.755791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade9fb-d2a8-4523-97ec-aaf3dce616d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:15.756081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade9fb-d2a8-4523-97ec-aaf3dce616d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:15.756764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ade9fb-d2a8-4523-97ec-aaf3dce616d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:15.760569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ade9fb-d2a8-4523-97ec-aaf3dce616d3 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:15.916929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1074/2170] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:55:15 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:55:15.918641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad0245-67a4-4f2b-bb14-f697db3e95c9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:15.933199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad0245-67a4-4f2b-bb14-f697db3e95c9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:15.943022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:16.032159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257165bf-d8f5-42d5-a25b-69d11eae1003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:16.033144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257165bf-d8f5-42d5-a25b-69d11eae1003 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:16.033602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257165bf-d8f5-42d5-a25b-69d11eae1003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:16.034029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257165bf-d8f5-42d5-a25b-69d11eae1003 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:16.034863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257165bf-d8f5-42d5-a25b-69d11eae1003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:16.037335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257165bf-d8f5-42d5-a25b-69d11eae1003 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:16.179275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1097/2171] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:55:15 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:16.188765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade9fb-d2a8-4523-97ec-aaf3dce616d3 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:16.199973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade9fb-d2a8-4523-97ec-aaf3dce616d3 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:16.205596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c82801-7722-40c0-9488-bcf3d4af6b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:16.206146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c82801-7722-40c0-9488-bcf3d4af6b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:16.206425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c82801-7722-40c0-9488-bcf3d4af6b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:16.206732 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c82801-7722-40c0-9488-bcf3d4af6b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:16.207386 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c82801-7722-40c0-9488-bcf3d4af6b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:16.209297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c82801-7722-40c0-9488-bcf3d4af6b70 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:16.220074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1075/2172] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:16.224416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257165bf-d8f5-42d5-a25b-69d11eae1003 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:16.234866 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257165bf-d8f5-42d5-a25b-69d11eae1003 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:16.240489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408aada0-2813-4d87-aba5-5d54789ca234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:16.241415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aada0-2813-4d87-aba5-5d54789ca234 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:16.241768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aada0-2813-4d87-aba5-5d54789ca234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:16.242141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aada0-2813-4d87-aba5-5d54789ca234 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:16.242899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408aada0-2813-4d87-aba5-5d54789ca234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:16.244639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408aada0-2813-4d87-aba5-5d54789ca234 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:16.256952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1098/2173] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:16.261654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c82801-7722-40c0-9488-bcf3d4af6b70 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:16.274146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c82801-7722-40c0-9488-bcf3d4af6b70 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:16.288496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91877b9f-c106-4433-a34d-a959384f3849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:16.289186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91877b9f-c106-4433-a34d-a959384f3849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:16.289470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91877b9f-c106-4433-a34d-a959384f3849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:16.289749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91877b9f-c106-4433-a34d-a959384f3849 admin admin] PATH_INFO: `/v3/projects/5c4b851eef494a36869c27fffafd6b9c/users/00dabf116a924e429f739ce886946f82/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:16.306577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91877b9f-c106-4433-a34d-a959384f3849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c4b851eef494a36869c27fffafd6b9c, user_id=00dabf116a924e429f739ce886946f82, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:16.309410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91877b9f-c106-4433-a34d-a959384f3849 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:16.559620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1076/2174] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:16 2022] PUT /identity/v3/projects/5c4b851eef494a36869c27fffafd6b9c/users/00dabf116a924e429f739ce886946f82/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:16.572148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488c572-6ba7-48bf-8663-c473d9239ed9 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:16.573128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488c572-6ba7-48bf-8663-c473d9239ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:16.573417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488c572-6ba7-48bf-8663-c473d9239ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:16.602365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-7488c572-6ba7-48bf-8663-c473d9239ed9 None None] MFA Rules not processed for user `00dabf116a924e429f739ce886946f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:16.603670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7488c572-6ba7-48bf-8663-c473d9239ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:16.671060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1099/2175] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:16.682711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91877b9f-c106-4433-a34d-a959384f3849 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:16.690282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-91877b9f-c106-4433-a34d-a959384f3849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:16.762902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91877b9f-c106-4433-a34d-a959384f3849 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:16.781204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65681a-242a-4baf-acaf-ca7299e7be2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:16.783041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65681a-242a-4baf-acaf-ca7299e7be2e service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:16.783967 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65681a-242a-4baf-acaf-ca7299e7be2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:16.784694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65681a-242a-4baf-acaf-ca7299e7be2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:16.804392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65681a-242a-4baf-acaf-ca7299e7be2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:16.807215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65681a-242a-4baf-acaf-ca7299e7be2e service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:16.827137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1077/2176] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:16.857315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488c572-6ba7-48bf-8663-c473d9239ed9 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:16.941315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488c572-6ba7-48bf-8663-c473d9239ed9 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:16.948982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:16.949651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:16.949939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:16.950263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] PATH_INFO: `/v3/users/00dabf116a924e429f739ce886946f82` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:16.953323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00dabf116a924e429f739ce886946f82)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:16.955372 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:17.235408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00dabf116a924e429f739ce886946f82'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:17.236356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00dabf116a924e429f739ce886946f82'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:17.336264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00dabf116a924e429f739ce886946f82'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:17.348459 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00dabf116a924e429f739ce886946f82'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:17.365597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00dabf116a924e429f739ce886946f82'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:17.371971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1100/2177] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:55:16 2022] DELETE /identity/v3/users/00dabf116a924e429f739ce886946f82 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:17.377584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5fb73-dcd0-44bc-92eb-60d8f4cd137e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:17.378298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5fb73-dcd0-44bc-92eb-60d8f4cd137e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:17.378298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5fb73-dcd0-44bc-92eb-60d8f4cd137e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:17.412080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e1c5fb73-dcd0-44bc-92eb-60d8f4cd137e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:17.414108 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e1c5fb73-dcd0-44bc-92eb-60d8f4cd137e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:17.491567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1078/2178] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:17.502965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:17.504596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:17.571232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8cb82-31b8-4b69-8139-440d9dd0a9f2 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:17.572353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca187332-a970-499c-af59-dd609fc03792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:17.572961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca187332-a970-499c-af59-dd609fc03792 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:17.573242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca187332-a970-499c-af59-dd609fc03792 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:17.573518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca187332-a970-499c-af59-dd609fc03792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:17.584655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca187332-a970-499c-af59-dd609fc03792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:17.586166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca187332-a970-499c-af59-dd609fc03792 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:17.598261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1101/2179] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:18.434337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5fb73-dcd0-44bc-92eb-60d8f4cd137e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:18.435773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e1c5fb73-dcd0-44bc-92eb-60d8f4cd137e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:18.459248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5fb73-dcd0-44bc-92eb-60d8f4cd137e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:18.475056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:18.475919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:18.476461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:18.476879 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] PATH_INFO: `/v3/projects/5c4b851eef494a36869c27fffafd6b9c` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:18.478581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c4b851eef494a36869c27fffafd6b9c)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:18.480558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:18.496414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:18.593469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c4b851eef494a36869c27fffafd6b9c', 'request_id': 'req-4caa99d5-a507-4e23-9046-fe06018efe36'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:18.661836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c4b851eef494a36869c27fffafd6b9c', 'request_id': 'req-4caa99d5-a507-4e23-9046-fe06018efe36'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:18.669434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c4b851eef494a36869c27fffafd6b9c', 'request_id': 'req-4caa99d5-a507-4e23-9046-fe06018efe36'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:18.682760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4f88a-484e-4d3a-a381-c9e73a873c35 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:18.683335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4f88a-484e-4d3a-a381-c9e73a873c35 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:18.683430 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4f88a-484e-4d3a-a381-c9e73a873c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:18.687121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] The token cache is being invalidate because project 5c4b851eef494a36869c27fffafd6b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:55:18.687653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:18.688400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4caa99d5-a507-4e23-9046-fe06018efe36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:18.690107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1079/2180] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:55:18 2022] DELETE /identity/v3/projects/5c4b851eef494a36869c27fffafd6b9c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:18.694824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a7286-b33c-4841-bf09-4cbb02a752ab None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:18.695117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a7286-b33c-4841-bf09-4cbb02a752ab None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:18.695546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a7286-b33c-4841-bf09-4cbb02a752ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:18.708556 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-85c4f88a-484e-4d3a-a381-c9e73a873c35 None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:18.709790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-85c4f88a-484e-4d3a-a381-c9e73a873c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:18.726526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-013a7286-b33c-4841-bf09-4cbb02a752ab None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:18.727978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-013a7286-b33c-4841-bf09-4cbb02a752ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:18.773535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:18.790193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1102/2181] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:55:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:55:18.804554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4f88a-484e-4d3a-a381-c9e73a873c35 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:18.806237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-85c4f88a-484e-4d3a-a381-c9e73a873c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:18.819606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1080/2182] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:55:18.830040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2d6f9-199e-4629-b0ef-55d6592f4179 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:18.830346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2d6f9-199e-4629-b0ef-55d6592f4179 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:18.830777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2d6f9-199e-4629-b0ef-55d6592f4179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:18.853056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-3eb2d6f9-199e-4629-b0ef-55d6592f4179 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:18.854443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3eb2d6f9-199e-4629-b0ef-55d6592f4179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:18.865337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1081/2183] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:55:18.869516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2d6f9-199e-4629-b0ef-55d6592f4179 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:18.879505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2d6f9-199e-4629-b0ef-55d6592f4179 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:18.885633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e9fd3-a6e7-4b06-98dc-90a662f70485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:18.886246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e9fd3-a6e7-4b06-98dc-90a662f70485 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:18.886549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e9fd3-a6e7-4b06-98dc-90a662f70485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:18.886835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e9fd3-a6e7-4b06-98dc-90a662f70485 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:18.887997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e9fd3-a6e7-4b06-98dc-90a662f70485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:55:18.888363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e9fd3-a6e7-4b06-98dc-90a662f70485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:18.890182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e9fd3-a6e7-4b06-98dc-90a662f70485 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:18.890891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4f88a-484e-4d3a-a381-c9e73a873c35 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:18.897349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445ba37-7b11-44e0-9fce-b1a9066ef9f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:18.898047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445ba37-7b11-44e0-9fce-b1a9066ef9f3 service swift] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:18.898599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445ba37-7b11-44e0-9fce-b1a9066ef9f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:18.899082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445ba37-7b11-44e0-9fce-b1a9066ef9f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:18.910676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1082/2184] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:55:18.910867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445ba37-7b11-44e0-9fce-b1a9066ef9f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:18.910867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445ba37-7b11-44e0-9fce-b1a9066ef9f3 service swift] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:18.912165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e9fd3-a6e7-4b06-98dc-90a662f70485 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:18.920267 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1103/2185] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:55:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:55:18.925858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e9fd3-a6e7-4b06-98dc-90a662f70485 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:18.936283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f219ee0-2ba4-4edc-a502-031ac041f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:18.936283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f219ee0-2ba4-4edc-a502-031ac041f10e admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:18.936283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f219ee0-2ba4-4edc-a502-031ac041f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:18.936283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f219ee0-2ba4-4edc-a502-031ac041f10e admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:18.936772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f219ee0-2ba4-4edc-a502-031ac041f10e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:18.936772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f219ee0-2ba4-4edc-a502-031ac041f10e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:19.083327 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1083/2186] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:55:18 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:55:19.088910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445ba37-7b11-44e0-9fce-b1a9066ef9f3 service swift] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:19.107224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445ba37-7b11-44e0-9fce-b1a9066ef9f3 service swift] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:19.209140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44102aee-9bed-4bab-9957-4b89e8216528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:19.209929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44102aee-9bed-4bab-9957-4b89e8216528 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:19.210320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44102aee-9bed-4bab-9957-4b89e8216528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:19.210706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44102aee-9bed-4bab-9957-4b89e8216528 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:19.211774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44102aee-9bed-4bab-9957-4b89e8216528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:19.213927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44102aee-9bed-4bab-9957-4b89e8216528 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:19.313187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f219ee0-2ba4-4edc-a502-031ac041f10e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:19.313760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1104/2187] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:55:19 2022] POST /identity/v3/users => generated 288 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:55:19.324742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f219ee0-2ba4-4edc-a502-031ac041f10e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:19.331512 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369afbe-d6d2-4475-9cdb-80d9eae3e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:19.332225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369afbe-d6d2-4475-9cdb-80d9eae3e259 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:19.332548 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369afbe-d6d2-4475-9cdb-80d9eae3e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:19.332896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369afbe-d6d2-4475-9cdb-80d9eae3e259 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:19.333693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369afbe-d6d2-4475-9cdb-80d9eae3e259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:19.337220 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369afbe-d6d2-4475-9cdb-80d9eae3e259 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:19.363835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1084/2188] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:19.370298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44102aee-9bed-4bab-9957-4b89e8216528 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:19.390943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44102aee-9bed-4bab-9957-4b89e8216528 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:19.407589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8fa7-6a68-44ab-8750-1b74e2ba8c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:19.409977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8fa7-6a68-44ab-8750-1b74e2ba8c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:19.410904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8fa7-6a68-44ab-8750-1b74e2ba8c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:19.412278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8fa7-6a68-44ab-8750-1b74e2ba8c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:19.413257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8fa7-6a68-44ab-8750-1b74e2ba8c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:19.439939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8fa7-6a68-44ab-8750-1b74e2ba8c83 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:19.484666 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1105/2189] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:19.486076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369afbe-d6d2-4475-9cdb-80d9eae3e259 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:19.502418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369afbe-d6d2-4475-9cdb-80d9eae3e259 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:19.518945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9575f5-77c0-420b-b205-f7ab0c89be01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:19.519502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9575f5-77c0-420b-b205-f7ab0c89be01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:19.519787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9575f5-77c0-420b-b205-f7ab0c89be01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:19.520065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9575f5-77c0-420b-b205-f7ab0c89be01 admin admin] PATH_INFO: `/v3/projects/e0a715442ed5463b8f25568027aa81b8/users/ce09fdc1e3554111999ebf748aa62b45/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:19.530386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:19.549909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9575f5-77c0-420b-b205-f7ab0c89be01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a715442ed5463b8f25568027aa81b8, user_id=ce09fdc1e3554111999ebf748aa62b45, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:19.552650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9575f5-77c0-420b-b205-f7ab0c89be01 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:19.630087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1085/2190] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:19 2022] PUT /identity/v3/projects/e0a715442ed5463b8f25568027aa81b8/users/ce09fdc1e3554111999ebf748aa62b45/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:19.641721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc146e9f-0082-407a-bc49-0e0573bb39fc None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:19.642389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc146e9f-0082-407a-bc49-0e0573bb39fc None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:19.642859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc146e9f-0082-407a-bc49-0e0573bb39fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:19.685448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-dc146e9f-0082-407a-bc49-0e0573bb39fc None None] MFA Rules not processed for user `ce09fdc1e3554111999ebf748aa62b45`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:19.688216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-dc146e9f-0082-407a-bc49-0e0573bb39fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:19.815083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1106/2191] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:19.823061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9575f5-77c0-420b-b205-f7ab0c89be01 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:19.925580 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9575f5-77c0-420b-b205-f7ab0c89be01 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:19.940562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5102f75-2456-4fb9-b4ad-3e20c2e3d683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:19.941582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5102f75-2456-4fb9-b4ad-3e20c2e3d683 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:19.942855 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5102f75-2456-4fb9-b4ad-3e20c2e3d683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:19.943745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5102f75-2456-4fb9-b4ad-3e20c2e3d683 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:19.945852 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5102f75-2456-4fb9-b4ad-3e20c2e3d683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:19.951611 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5102f75-2456-4fb9-b4ad-3e20c2e3d683 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:20.297504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:20.381836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1086/2192] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:55:19 2022] POST /identity/v3/projects => generated 375 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:20.385021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc146e9f-0082-407a-bc49-0e0573bb39fc None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:20.431801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc146e9f-0082-407a-bc49-0e0573bb39fc None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:20.507724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:20.592236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58a09e-0208-400e-8399-2d6c51a08a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:20.593112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58a09e-0208-400e-8399-2d6c51a08a67 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:20.593404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58a09e-0208-400e-8399-2d6c51a08a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:20.593801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58a09e-0208-400e-8399-2d6c51a08a67 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:20.611420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58a09e-0208-400e-8399-2d6c51a08a67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:20.613983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58a09e-0208-400e-8399-2d6c51a08a67 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:20.779031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1107/2193] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:55:20 2022] POST /identity/v3/users => generated 288 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:55:20.783824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5102f75-2456-4fb9-b4ad-3e20c2e3d683 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:20.809294 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5102f75-2456-4fb9-b4ad-3e20c2e3d683 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:20.825741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d473d-2120-4983-9ba9-e44c95f92d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:20.826690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d473d-2120-4983-9ba9-e44c95f92d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:20.827140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d473d-2120-4983-9ba9-e44c95f92d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:20.827584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d473d-2120-4983-9ba9-e44c95f92d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:20.828401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d473d-2120-4983-9ba9-e44c95f92d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:20.833444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d473d-2120-4983-9ba9-e44c95f92d41 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:20.867476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1087/2194] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:20.868626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58a09e-0208-400e-8399-2d6c51a08a67 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:20.892755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58a09e-0208-400e-8399-2d6c51a08a67 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:20.904124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3ac04-d12f-4174-91cf-c8691ffb5c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:20.905661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3ac04-d12f-4174-91cf-c8691ffb5c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:20.906260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3ac04-d12f-4174-91cf-c8691ffb5c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:20.907080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3ac04-d12f-4174-91cf-c8691ffb5c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:20.908295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3ac04-d12f-4174-91cf-c8691ffb5c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:20.912169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3ac04-d12f-4174-91cf-c8691ffb5c4a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:20.933438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1108/2195] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:20.937993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d473d-2120-4983-9ba9-e44c95f92d41 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:20.961928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d473d-2120-4983-9ba9-e44c95f92d41 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:20.982177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e688-61da-4c45-8c3d-e386758e527a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:20.984808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e688-61da-4c45-8c3d-e386758e527a admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:20.986634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e688-61da-4c45-8c3d-e386758e527a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:20.987412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e688-61da-4c45-8c3d-e386758e527a admin admin] PATH_INFO: `/v3/projects/ede311434d7441279bd44665e3a0e35b/users/0443b95b5dac4e1bb7569e3e4e8aca19/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:21.031406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e688-61da-4c45-8c3d-e386758e527a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede311434d7441279bd44665e3a0e35b, user_id=0443b95b5dac4e1bb7569e3e4e8aca19, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:21.034937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e688-61da-4c45-8c3d-e386758e527a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:21.145672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1088/2196] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:20 2022] PUT /identity/v3/projects/ede311434d7441279bd44665e3a0e35b/users/0443b95b5dac4e1bb7569e3e4e8aca19/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:21.148471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05c781-d2c9-4e4b-aa8a-da277d21953c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:21.148766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05c781-d2c9-4e4b-aa8a-da277d21953c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:21.149050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05c781-d2c9-4e4b-aa8a-da277d21953c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:21.183695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-2d05c781-d2c9-4e4b-aa8a-da277d21953c None None] MFA Rules not processed for user `0443b95b5dac4e1bb7569e3e4e8aca19`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:21.187200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2d05c781-d2c9-4e4b-aa8a-da277d21953c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:21.269816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1109/2197] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:21.271047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e688-61da-4c45-8c3d-e386758e527a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:21.359268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e688-61da-4c45-8c3d-e386758e527a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:21.365307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f0daa-a922-4e0b-8cd8-55a901911ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:21.366290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f0daa-a922-4e0b-8cd8-55a901911ffb admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:21.367324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f0daa-a922-4e0b-8cd8-55a901911ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:21.367609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f0daa-a922-4e0b-8cd8-55a901911ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:21.368691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f0daa-a922-4e0b-8cd8-55a901911ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:21.370214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f0daa-a922-4e0b-8cd8-55a901911ffb admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:21.504873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1089/2198] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:55:21 2022] POST /identity/v3/projects => generated 377 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:21.509318 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05c781-d2c9-4e4b-aa8a-da277d21953c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:21.522242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05c781-d2c9-4e4b-aa8a-da277d21953c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:21.585528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10aa50-6feb-4209-94ce-77c929af4a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:21.586280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10aa50-6feb-4209-94ce-77c929af4a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:21.586658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10aa50-6feb-4209-94ce-77c929af4a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:21.586860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10aa50-6feb-4209-94ce-77c929af4a5e admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:21.587601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10aa50-6feb-4209-94ce-77c929af4a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:21.589573 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10aa50-6feb-4209-94ce-77c929af4a5e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:21.602340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:21.738146 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1110/2199] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:55:21 2022] POST /identity/v3/users => generated 295 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:55:21.742227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f0daa-a922-4e0b-8cd8-55a901911ffb admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:21.756872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f0daa-a922-4e0b-8cd8-55a901911ffb admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:21.763500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00610523-a8a8-4e2b-b628-2e948c6b9e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:21.764470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00610523-a8a8-4e2b-b628-2e948c6b9e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:21.764819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00610523-a8a8-4e2b-b628-2e948c6b9e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:21.765094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00610523-a8a8-4e2b-b628-2e948c6b9e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:21.766034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00610523-a8a8-4e2b-b628-2e948c6b9e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:21.769786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00610523-a8a8-4e2b-b628-2e948c6b9e49 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:21.805646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1090/2200] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:21.812267 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10aa50-6feb-4209-94ce-77c929af4a5e admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:21.826752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10aa50-6feb-4209-94ce-77c929af4a5e admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:21.832064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522eb46-547f-469e-b0ad-d219131391e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:21.832829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522eb46-547f-469e-b0ad-d219131391e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:21.833113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522eb46-547f-469e-b0ad-d219131391e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:21.833389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522eb46-547f-469e-b0ad-d219131391e9 admin admin] PATH_INFO: `/v3/projects/eb2690a2b6f9408aa4559803bf5a2e7b/users/3cf99d06ba29444a981989ce77fb51be/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:21.849659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522eb46-547f-469e-b0ad-d219131391e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2690a2b6f9408aa4559803bf5a2e7b, user_id=3cf99d06ba29444a981989ce77fb51be, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:21.852554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522eb46-547f-469e-b0ad-d219131391e9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:22.258218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1111/2201] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:21 2022] PUT /identity/v3/projects/eb2690a2b6f9408aa4559803bf5a2e7b/users/3cf99d06ba29444a981989ce77fb51be/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:22.262565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00610523-a8a8-4e2b-b628-2e948c6b9e49 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:22.279105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00610523-a8a8-4e2b-b628-2e948c6b9e49 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:22.349248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075b1f-050a-4a8b-940a-44aad769ba05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:22.350282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075b1f-050a-4a8b-940a-44aad769ba05 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:22.350670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075b1f-050a-4a8b-940a-44aad769ba05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:22.350925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075b1f-050a-4a8b-940a-44aad769ba05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:22.351638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075b1f-050a-4a8b-940a-44aad769ba05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:22.353468 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075b1f-050a-4a8b-940a-44aad769ba05 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:22.365476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1091/2202] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:22.369795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522eb46-547f-469e-b0ad-d219131391e9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:22.384985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522eb46-547f-469e-b0ad-d219131391e9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:22.391922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedafb95-e61e-48ae-8f8e-71cdbeac2d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:22.393045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedafb95-e61e-48ae-8f8e-71cdbeac2d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:22.393328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedafb95-e61e-48ae-8f8e-71cdbeac2d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:22.393750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedafb95-e61e-48ae-8f8e-71cdbeac2d3f admin admin] PATH_INFO: `/v3/projects/eb2690a2b6f9408aa4559803bf5a2e7b/users/3cf99d06ba29444a981989ce77fb51be/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:22.395910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedafb95-e61e-48ae-8f8e-71cdbeac2d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2690a2b6f9408aa4559803bf5a2e7b, user_id=3cf99d06ba29444a981989ce77fb51be, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:22.399110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedafb95-e61e-48ae-8f8e-71cdbeac2d3f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:22.422542 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1112/2203] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:22 2022] PUT /identity/v3/projects/eb2690a2b6f9408aa4559803bf5a2e7b/users/3cf99d06ba29444a981989ce77fb51be/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:55:22.426500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6124bb-df73-46f5-ae69-b84c1532e8cb None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:22.426950 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6124bb-df73-46f5-ae69-b84c1532e8cb None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:22.427329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6124bb-df73-46f5-ae69-b84c1532e8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:22.459154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:22.470076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6c6124bb-df73-46f5-ae69-b84c1532e8cb None None] MFA Rules not processed for user `3cf99d06ba29444a981989ce77fb51be`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:22.472570 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6c6124bb-df73-46f5-ae69-b84c1532e8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:22.586978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1092/2204] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:22.587918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedafb95-e61e-48ae-8f8e-71cdbeac2d3f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:22.589666 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-cedafb95-e61e-48ae-8f8e-71cdbeac2d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:22.677856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedafb95-e61e-48ae-8f8e-71cdbeac2d3f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:22.681599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d427155-09da-4fef-a68e-dda600c891b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:22.682220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d427155-09da-4fef-a68e-dda600c891b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:22.682559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d427155-09da-4fef-a68e-dda600c891b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:22.682821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d427155-09da-4fef-a68e-dda600c891b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:22.814448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d427155-09da-4fef-a68e-dda600c891b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:22.816086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d427155-09da-4fef-a68e-dda600c891b5 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:22.827311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1113/2205] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:55:26.107894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6124bb-df73-46f5-ae69-b84c1532e8cb None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:26.120831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6124bb-df73-46f5-ae69-b84c1532e8cb None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:26.127194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf57f6d-a7da-461e-8a71-8e463e21776b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:26.127858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf57f6d-a7da-461e-8a71-8e463e21776b service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:26.128143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf57f6d-a7da-461e-8a71-8e463e21776b service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:26.128481 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf57f6d-a7da-461e-8a71-8e463e21776b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:26.207600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf57f6d-a7da-461e-8a71-8e463e21776b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:26.209082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf57f6d-a7da-461e-8a71-8e463e21776b service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:26.219272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1093/2206] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:28.125346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d427155-09da-4fef-a68e-dda600c891b5 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:28.127032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2d427155-09da-4fef-a68e-dda600c891b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:28.139311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d427155-09da-4fef-a68e-dda600c891b5 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:28.143296 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab7f17-5377-46b0-bd69-a7b650342887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:28.144002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab7f17-5377-46b0-bd69-a7b650342887 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:28.144380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab7f17-5377-46b0-bd69-a7b650342887 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:28.144760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab7f17-5377-46b0-bd69-a7b650342887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:28.155386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab7f17-5377-46b0-bd69-a7b650342887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:28.156958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab7f17-5377-46b0-bd69-a7b650342887 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:28.168307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1114/2207] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:32.664232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf57f6d-a7da-461e-8a71-8e463e21776b service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:32.665643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2bf57f6d-a7da-461e-8a71-8e463e21776b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:32.724434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf57f6d-a7da-461e-8a71-8e463e21776b service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:32.728182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019f91b-0e18-451f-8262-fd521df29a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:32.728795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019f91b-0e18-451f-8262-fd521df29a1f service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:32.729080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019f91b-0e18-451f-8262-fd521df29a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:32.729359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019f91b-0e18-451f-8262-fd521df29a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:32.732033 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7019f91b-0e18-451f-8262-fd521df29a1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:32.783000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:32.799831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019f91b-0e18-451f-8262-fd521df29a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:32.801359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019f91b-0e18-451f-8262-fd521df29a1f service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:32.811035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1094/2208] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:32.977796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab7f17-5377-46b0-bd69-a7b650342887 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:33.040616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:33.045800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab7f17-5377-46b0-bd69-a7b650342887 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:33.050376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:33.051073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:33.051355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:33.051766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] PATH_INFO: `/v3/users/ce09fdc1e3554111999ebf748aa62b45` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:33.052912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce09fdc1e3554111999ebf748aa62b45)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:33.055011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:33.333218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce09fdc1e3554111999ebf748aa62b45'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:33.334178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce09fdc1e3554111999ebf748aa62b45'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:33.402585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce09fdc1e3554111999ebf748aa62b45'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:33.410349 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce09fdc1e3554111999ebf748aa62b45'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:33.422293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce09fdc1e3554111999ebf748aa62b45'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:33.428233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1115/2209] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:55:32 2022] DELETE /identity/v3/users/ce09fdc1e3554111999ebf748aa62b45 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:33.432964 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645af2e-85e4-4526-86b8-0aa5fff89099 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:33.433251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645af2e-85e4-4526-86b8-0aa5fff89099 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:33.433531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645af2e-85e4-4526-86b8-0aa5fff89099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:33.453835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7645af2e-85e4-4526-86b8-0aa5fff89099 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:33.455086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7645af2e-85e4-4526-86b8-0aa5fff89099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:33.516911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1095/2210] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:55:33.528884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:33.530578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:33.591117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270ea4f-a113-4a1d-9d00-f081a3c4b5de admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:33.594814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebb0f7-8247-4822-b3c7-eef7ed12ce8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:33.595422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebb0f7-8247-4822-b3c7-eef7ed12ce8a service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:33.595752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebb0f7-8247-4822-b3c7-eef7ed12ce8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:33.596016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebb0f7-8247-4822-b3c7-eef7ed12ce8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:33.606330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebb0f7-8247-4822-b3c7-eef7ed12ce8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:33.608023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebb0f7-8247-4822-b3c7-eef7ed12ce8a service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:33.617879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1116/2211] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:34.038213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645af2e-85e4-4526-86b8-0aa5fff89099 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:34.040440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7645af2e-85e4-4526-86b8-0aa5fff89099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:34.065679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645af2e-85e4-4526-86b8-0aa5fff89099 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:34.069541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:34.070187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:34.070708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:34.071139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] PATH_INFO: `/v3/projects/e0a715442ed5463b8f25568027aa81b8` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:34.072306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a715442ed5463b8f25568027aa81b8)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:34.074217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:34.209513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a715442ed5463b8f25568027aa81b8', 'request_id': 'req-14387643-561f-44a1-910e-deb3fa6953ff'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.316881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a715442ed5463b8f25568027aa81b8', 'request_id': 'req-14387643-561f-44a1-910e-deb3fa6953ff'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.324171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a715442ed5463b8f25568027aa81b8', 'request_id': 'req-14387643-561f-44a1-910e-deb3fa6953ff'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.336642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] The token cache is being invalidate because project e0a715442ed5463b8f25568027aa81b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:55:34.339417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.339417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.339417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1096/2212] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:55:34 2022] DELETE /identity/v3/projects/e0a715442ed5463b8f25568027aa81b8 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:55:34.343946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebb0f7-8247-4822-b3c7-eef7ed12ce8a service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:34.353599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebb0f7-8247-4822-b3c7-eef7ed12ce8a service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:34.420783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:34.421423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:34.421707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:34.422039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] PATH_INFO: `/v3/users/0443b95b5dac4e1bb7569e3e4e8aca19` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:34.423212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0443b95b5dac4e1bb7569e3e4e8aca19)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:34.425179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:34.450985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:34.630316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0443b95b5dac4e1bb7569e3e4e8aca19'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.630922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0443b95b5dac4e1bb7569e3e4e8aca19'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.682880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0443b95b5dac4e1bb7569e3e4e8aca19'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.691298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0443b95b5dac4e1bb7569e3e4e8aca19'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.704425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0443b95b5dac4e1bb7569e3e4e8aca19'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:34.712462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1117/2213] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:55:34 2022] DELETE /identity/v3/users/0443b95b5dac4e1bb7569e3e4e8aca19 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:35.126168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:35.132211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:35.151259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14387643-561f-44a1-910e-deb3fa6953ff admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:35.220741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:35.221856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:35.221991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:35.222327 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] PATH_INFO: `/v3/projects/ede311434d7441279bd44665e3a0e35b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:35.223797 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede311434d7441279bd44665e3a0e35b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:35.226873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:35.239783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:35.379703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede311434d7441279bd44665e3a0e35b', 'request_id': 'req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.454388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede311434d7441279bd44665e3a0e35b', 'request_id': 'req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.463957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede311434d7441279bd44665e3a0e35b', 'request_id': 'req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.478194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] The token cache is being invalidate because project ede311434d7441279bd44665e3a0e35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:55:35.478717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.479395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.485595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1097/2214] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:55:35 2022] DELETE /identity/v3/projects/ede311434d7441279bd44665e3a0e35b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:35.486654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:35.488164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:35.501106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feaa5d9-af65-436e-aa58-c65f5029ca53 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:35.567781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:35.568367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:35.568757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:35.569132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] PATH_INFO: `/v3/users/3cf99d06ba29444a981989ce77fb51be` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:35.570991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf99d06ba29444a981989ce77fb51be)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:35.573024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:35.832986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf99d06ba29444a981989ce77fb51be'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.833676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf99d06ba29444a981989ce77fb51be'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.911124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf99d06ba29444a981989ce77fb51be'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.920154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf99d06ba29444a981989ce77fb51be'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.933160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-36d1b2a4-4f13-473b-a6db-12ce609b9f66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf99d06ba29444a981989ce77fb51be'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.939613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1118/2215] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:55:35 2022] DELETE /identity/v3/users/3cf99d06ba29444a981989ce77fb51be => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:36.342432 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:36.344301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:36.363603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7f991-239b-4c3f-9a4d-3480c0fabe48 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:36.426970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:36.427632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:36.427913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:36.428191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] PATH_INFO: `/v3/projects/eb2690a2b6f9408aa4559803bf5a2e7b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:36.429364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb2690a2b6f9408aa4559803bf5a2e7b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:36.431440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:36.581384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb2690a2b6f9408aa4559803bf5a2e7b', 'request_id': 'req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:36.687925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb2690a2b6f9408aa4559803bf5a2e7b', 'request_id': 'req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:36.695493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb2690a2b6f9408aa4559803bf5a2e7b', 'request_id': 'req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:36.708610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] The token cache is being invalidate because project eb2690a2b6f9408aa4559803bf5a2e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:55:36.708895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:36.709557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-12d47a4a-11dd-4ebe-85bb-3ee43be0c968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:36.711454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1098/2216] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:55:36 2022] DELETE /identity/v3/projects/eb2690a2b6f9408aa4559803bf5a2e7b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:36.717224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6d716-db85-4742-a7c5-91a50fbdc2cf None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:36.717224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6d716-db85-4742-a7c5-91a50fbdc2cf None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:36.717771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6d716-db85-4742-a7c5-91a50fbdc2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:36.736892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-14e6d716-db85-4742-a7c5-91a50fbdc2cf None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:36.738036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-14e6d716-db85-4742-a7c5-91a50fbdc2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:36.750254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:36.798391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1119/2217] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:55:36.806483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb794069-8683-4251-9603-41b55fd04af6 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:36.806825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb794069-8683-4251-9603-41b55fd04af6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:36.807669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb794069-8683-4251-9603-41b55fd04af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:36.829238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-cb794069-8683-4251-9603-41b55fd04af6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:36.830453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cb794069-8683-4251-9603-41b55fd04af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:36.838672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1099/2218] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:36.843341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6d716-db85-4742-a7c5-91a50fbdc2cf None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:36.853242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6d716-db85-4742-a7c5-91a50fbdc2cf None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:36.860029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14131255-44a2-41ea-bcc8-39d2a5956246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:36.862650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14131255-44a2-41ea-bcc8-39d2a5956246 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:36.862650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14131255-44a2-41ea-bcc8-39d2a5956246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:36.862650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14131255-44a2-41ea-bcc8-39d2a5956246 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:36.862650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14131255-44a2-41ea-bcc8-39d2a5956246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:55:36.862650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14131255-44a2-41ea-bcc8-39d2a5956246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:36.864176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14131255-44a2-41ea-bcc8-39d2a5956246 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:36.880494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1120/2219] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:55:36.881584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb794069-8683-4251-9603-41b55fd04af6 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:36.892800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb794069-8683-4251-9603-41b55fd04af6 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:36.904632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f146323-6050-4516-bc67-e3f826cfb9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:36.906252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f146323-6050-4516-bc67-e3f826cfb9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:36.906658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f146323-6050-4516-bc67-e3f826cfb9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:36.907011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f146323-6050-4516-bc67-e3f826cfb9b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:36.907810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f146323-6050-4516-bc67-e3f826cfb9b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:36.910984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f146323-6050-4516-bc67-e3f826cfb9b4 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:36.997555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:37.007960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1100/2220] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:55:36 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:55:37.009104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14131255-44a2-41ea-bcc8-39d2a5956246 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:37.025586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14131255-44a2-41ea-bcc8-39d2a5956246 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:37.089117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0529cd5-77a5-48e3-aa52-d0b920a05cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:37.089763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0529cd5-77a5-48e3-aa52-d0b920a05cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:37.090631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0529cd5-77a5-48e3-aa52-d0b920a05cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:37.090906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0529cd5-77a5-48e3-aa52-d0b920a05cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:37.091541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0529cd5-77a5-48e3-aa52-d0b920a05cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:37.093544 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0529cd5-77a5-48e3-aa52-d0b920a05cd6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:37.220439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1121/2221] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:55:37 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:55:37.224646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f146323-6050-4516-bc67-e3f826cfb9b4 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:37.235871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f146323-6050-4516-bc67-e3f826cfb9b4 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:37.240356 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a12ff-7d47-4d79-b3ee-b6a28bdb770e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:37.240978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a12ff-7d47-4d79-b3ee-b6a28bdb770e admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:37.241262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a12ff-7d47-4d79-b3ee-b6a28bdb770e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:37.241538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a12ff-7d47-4d79-b3ee-b6a28bdb770e admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:37.242185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a12ff-7d47-4d79-b3ee-b6a28bdb770e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:37.243842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a12ff-7d47-4d79-b3ee-b6a28bdb770e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:37.254553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1101/2222] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:55:37.258673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0529cd5-77a5-48e3-aa52-d0b920a05cd6 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:37.272790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0529cd5-77a5-48e3-aa52-d0b920a05cd6 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:37.279120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d44969-caea-460b-aff1-24af330947ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:37.279975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d44969-caea-460b-aff1-24af330947ee admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:37.280688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d44969-caea-460b-aff1-24af330947ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:37.280688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d44969-caea-460b-aff1-24af330947ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:37.281301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d44969-caea-460b-aff1-24af330947ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:37.283096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d44969-caea-460b-aff1-24af330947ee admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:37.296518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1122/2223] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:55:37.300627 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a12ff-7d47-4d79-b3ee-b6a28bdb770e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:37.311212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a12ff-7d47-4d79-b3ee-b6a28bdb770e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:37.315717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d32dc9-9f2b-4713-bd93-da4119deb2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:37.316370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d32dc9-9f2b-4713-bd93-da4119deb2f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:37.316652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d32dc9-9f2b-4713-bd93-da4119deb2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:37.316931 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d32dc9-9f2b-4713-bd93-da4119deb2f9 admin admin] PATH_INFO: `/v3/projects/be4356bdf4c34660b22685d504344ad6/users/224f3b0b68b3410cbe74d9a6f822d6b9/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:37.332733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d32dc9-9f2b-4713-bd93-da4119deb2f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be4356bdf4c34660b22685d504344ad6, user_id=224f3b0b68b3410cbe74d9a6f822d6b9, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:37.335489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d32dc9-9f2b-4713-bd93-da4119deb2f9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:37.472450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1102/2224] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:55:37 2022] PUT /identity/v3/projects/be4356bdf4c34660b22685d504344ad6/users/224f3b0b68b3410cbe74d9a6f822d6b9/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:37.541544 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dafe9f-1820-4478-962c-32d78872c875 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:37.541836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dafe9f-1820-4478-962c-32d78872c875 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:37.542114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dafe9f-1820-4478-962c-32d78872c875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:37.571045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a5dafe9f-1820-4478-962c-32d78872c875 None None] MFA Rules not processed for user `224f3b0b68b3410cbe74d9a6f822d6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:37.572285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a5dafe9f-1820-4478-962c-32d78872c875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:37.646631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1123/2225] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:55:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:37.659355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d32dc9-9f2b-4713-bd93-da4119deb2f9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:37.661089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-93d32dc9-9f2b-4713-bd93-da4119deb2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:37.724733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d32dc9-9f2b-4713-bd93-da4119deb2f9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:37.728833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1ce9e-c486-4863-84b9-e87a3a50f238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:37.729442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1ce9e-c486-4863-84b9-e87a3a50f238 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:37.729724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1ce9e-c486-4863-84b9-e87a3a50f238 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:37.729998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1ce9e-c486-4863-84b9-e87a3a50f238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:37.740110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1ce9e-c486-4863-84b9-e87a3a50f238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:37.741082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1ce9e-c486-4863-84b9-e87a3a50f238 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:37.756711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1103/2226] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:44.210676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff02c0-696a-44bf-bb7f-6774d2c6a07d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:44.211162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff02c0-696a-44bf-bb7f-6774d2c6a07d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:44.211344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff02c0-696a-44bf-bb7f-6774d2c6a07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:44.231013 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e1ff02c0-696a-44bf-bb7f-6774d2c6a07d None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:44.232225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e1ff02c0-696a-44bf-bb7f-6774d2c6a07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:44.240574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:44.286296 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1124/2227] 10.222.0.82 () {58 vars in 967 bytes} [Wed Aug 24 03:55:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:44.308442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1ce9e-c486-4863-84b9-e87a3a50f238 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:44.309904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8ec1ce9e-c486-4863-84b9-e87a3a50f238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:44.381173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1ce9e-c486-4863-84b9-e87a3a50f238 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:44.387341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369bff7-35c2-4fa8-a3b7-b34fe081022b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:44.387954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369bff7-35c2-4fa8-a3b7-b34fe081022b service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:44.388239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369bff7-35c2-4fa8-a3b7-b34fe081022b service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:44.388530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369bff7-35c2-4fa8-a3b7-b34fe081022b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:44.397388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369bff7-35c2-4fa8-a3b7-b34fe081022b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:44.399144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369bff7-35c2-4fa8-a3b7-b34fe081022b service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:44.408629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1104/2228] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:55:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:55:54.361503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=19 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:54.409265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:56:23.023392 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff02c0-696a-44bf-bb7f-6774d2c6a07d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:23.093385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff02c0-696a-44bf-bb7f-6774d2c6a07d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:23.094717 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:23.095363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:23.095626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:23.096000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] PATH_INFO: `/v3/users/224f3b0b68b3410cbe74d9a6f822d6b9` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:23.097229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=224f3b0b68b3410cbe74d9a6f822d6b9)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:23.099136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:23.334906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224f3b0b68b3410cbe74d9a6f822d6b9'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:23.335577 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224f3b0b68b3410cbe74d9a6f822d6b9'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:23.409993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224f3b0b68b3410cbe74d9a6f822d6b9'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:23.418002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224f3b0b68b3410cbe74d9a6f822d6b9'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:23.429950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224f3b0b68b3410cbe74d9a6f822d6b9'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:23.436375 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1125/2229] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:56:23 2022] DELETE /identity/v3/users/224f3b0b68b3410cbe74d9a6f822d6b9 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:56:23.441095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4f13e-bed6-4aaf-8ffd-8d2610ebc4d6 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:23.446838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4f13e-bed6-4aaf-8ffd-8d2610ebc4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:23.446838 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4f13e-bed6-4aaf-8ffd-8d2610ebc4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:23.463354 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6dc4f13e-bed6-4aaf-8ffd-8d2610ebc4d6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:56:23.464609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6dc4f13e-bed6-4aaf-8ffd-8d2610ebc4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:23.531799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1105/2230] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:56:23.541617 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:23.543080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:23.600501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285e3d2-c0f2-4c5f-b009-9c43bc79f8ef admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:23.604193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f20869-e61c-47ae-a1b3-237ff512053f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:23.604799 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f20869-e61c-47ae-a1b3-237ff512053f service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:23.605081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f20869-e61c-47ae-a1b3-237ff512053f service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:23.605358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f20869-e61c-47ae-a1b3-237ff512053f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:23.614264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f20869-e61c-47ae-a1b3-237ff512053f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:23.615904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f20869-e61c-47ae-a1b3-237ff512053f service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:23.625773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1126/2231] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:56:24.043265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4f13e-bed6-4aaf-8ffd-8d2610ebc4d6 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:24.044975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6dc4f13e-bed6-4aaf-8ffd-8d2610ebc4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:24.068614 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4f13e-bed6-4aaf-8ffd-8d2610ebc4d6 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:24.076209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:24.077045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.077384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.077671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] PATH_INFO: `/v3/projects/be4356bdf4c34660b22685d504344ad6` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.078982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be4356bdf4c34660b22685d504344ad6)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:24.081293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:24.192140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be4356bdf4c34660b22685d504344ad6', 'request_id': 'req-83cc14f1-683c-4c55-9f79-7c0699c4d713'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:24.261882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be4356bdf4c34660b22685d504344ad6', 'request_id': 'req-83cc14f1-683c-4c55-9f79-7c0699c4d713'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:24.269311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be4356bdf4c34660b22685d504344ad6', 'request_id': 'req-83cc14f1-683c-4c55-9f79-7c0699c4d713'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:24.289362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] The token cache is being invalidate because project be4356bdf4c34660b22685d504344ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:56:24.289362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:24.289362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-83cc14f1-683c-4c55-9f79-7c0699c4d713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:24.291898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1106/2232] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:56:24 2022] DELETE /identity/v3/projects/be4356bdf4c34660b22685d504344ad6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:56:24.295429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82660-bac8-4383-89b0-9a1dca652eee None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.296214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82660-bac8-4383-89b0-9a1dca652eee None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.296640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82660-bac8-4383-89b0-9a1dca652eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.319219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-3aa82660-bac8-4383-89b0-9a1dca652eee None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:56:24.320483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3aa82660-bac8-4383-89b0-9a1dca652eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:24.343272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:56:24.395500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1127/2233] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:56:24.404949 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3fc0f-8666-4336-bf21-6451e93f798a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.405148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3fc0f-8666-4336-bf21-6451e93f798a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.409948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3fc0f-8666-4336-bf21-6451e93f798a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.426261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-42f3fc0f-8666-4336-bf21-6451e93f798a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:56:24.427553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-42f3fc0f-8666-4336-bf21-6451e93f798a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:24.437296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1107/2234] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:56:24.441843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa82660-bac8-4383-89b0-9a1dca652eee None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:24.454884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa82660-bac8-4383-89b0-9a1dca652eee None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:24.460143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2aca8-0642-4a06-bcba-62cc57d6fa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:24.460836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2aca8-0642-4a06-bcba-62cc57d6fa34 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.461194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2aca8-0642-4a06-bcba-62cc57d6fa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.461542 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2aca8-0642-4a06-bcba-62cc57d6fa34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.462271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2aca8-0642-4a06-bcba-62cc57d6fa34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:56:24.463273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2aca8-0642-4a06-bcba-62cc57d6fa34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:24.465064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2aca8-0642-4a06-bcba-62cc57d6fa34 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:24.481444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1128/2235] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:56:24.482656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3fc0f-8666-4336-bf21-6451e93f798a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:24.492821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3fc0f-8666-4336-bf21-6451e93f798a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:24.498741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9a924-a55e-4932-80df-11859a61ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:24.499555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9a924-a55e-4932-80df-11859a61ff11 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.499843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9a924-a55e-4932-80df-11859a61ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.500120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9a924-a55e-4932-80df-11859a61ff11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.500864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9a924-a55e-4932-80df-11859a61ff11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:24.504858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9a924-a55e-4932-80df-11859a61ff11 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:24.589916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1108/2236] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:56:24 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:56:24.594176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2aca8-0642-4a06-bcba-62cc57d6fa34 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:24.604685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2aca8-0642-4a06-bcba-62cc57d6fa34 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:24.669451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b9e95-6b3d-41bf-b0e0-783b9f62fe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:24.670066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b9e95-6b3d-41bf-b0e0-783b9f62fe46 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.670345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b9e95-6b3d-41bf-b0e0-783b9f62fe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.670703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b9e95-6b3d-41bf-b0e0-783b9f62fe46 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.671338 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b9e95-6b3d-41bf-b0e0-783b9f62fe46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:24.673367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b9e95-6b3d-41bf-b0e0-783b9f62fe46 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:24.751065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1129/2237] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:56:24 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:56:24.752277 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9a924-a55e-4932-80df-11859a61ff11 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:24.768415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9a924-a55e-4932-80df-11859a61ff11 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:24.773452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe7583-656d-42a7-92a0-24aca757cdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:24.774052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe7583-656d-42a7-92a0-24aca757cdd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.774251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe7583-656d-42a7-92a0-24aca757cdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.774557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe7583-656d-42a7-92a0-24aca757cdd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.775305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe7583-656d-42a7-92a0-24aca757cdd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:24.776932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe7583-656d-42a7-92a0-24aca757cdd6 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:24.789919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1109/2238] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:56:24.793798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b9e95-6b3d-41bf-b0e0-783b9f62fe46 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:24.804554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b9e95-6b3d-41bf-b0e0-783b9f62fe46 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:24.811058 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1e403-763c-4832-9856-d930348e7a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:24.811810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1e403-763c-4832-9856-d930348e7a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.812501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1e403-763c-4832-9856-d930348e7a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.812501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1e403-763c-4832-9856-d930348e7a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.813225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1e403-763c-4832-9856-d930348e7a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:24.815056 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1e403-763c-4832-9856-d930348e7a3d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:24.831280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1130/2239] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:56:24.835254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe7583-656d-42a7-92a0-24aca757cdd6 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:24.845635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe7583-656d-42a7-92a0-24aca757cdd6 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:24.850674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303372c-232b-429d-b2ae-6281bb8bd9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:24.851334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303372c-232b-429d-b2ae-6281bb8bd9ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.851621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303372c-232b-429d-b2ae-6281bb8bd9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.851901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303372c-232b-429d-b2ae-6281bb8bd9ea admin admin] PATH_INFO: `/v3/projects/bb2bd7e390bd444cb2f9d819d8a2ef6a/users/a3970092c268469cae2a83d73d20f875/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.868084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303372c-232b-429d-b2ae-6281bb8bd9ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2bd7e390bd444cb2f9d819d8a2ef6a, user_id=a3970092c268469cae2a83d73d20f875, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:24.871581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303372c-232b-429d-b2ae-6281bb8bd9ea admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:24.930637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1110/2240] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:56:24 2022] PUT /identity/v3/projects/bb2bd7e390bd444cb2f9d819d8a2ef6a/users/a3970092c268469cae2a83d73d20f875/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:56:24.943155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0608-3094-4ef1-9e17-35ea774f10f1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:24.943684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0608-3094-4ef1-9e17-35ea774f10f1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:24.943963 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0608-3094-4ef1-9e17-35ea774f10f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:24.973620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-1c8c0608-3094-4ef1-9e17-35ea774f10f1 None None] MFA Rules not processed for user `a3970092c268469cae2a83d73d20f875`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:56:24.975675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1c8c0608-3094-4ef1-9e17-35ea774f10f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:25.053212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1131/2241] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:56:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:56:25.064692 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303372c-232b-429d-b2ae-6281bb8bd9ea admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:25.066673 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8303372c-232b-429d-b2ae-6281bb8bd9ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:25.091112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:56:25.130072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303372c-232b-429d-b2ae-6281bb8bd9ea admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:25.133800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b99e8-91b1-4b79-84e6-791b20a539fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:25.134993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b99e8-91b1-4b79-84e6-791b20a539fb service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:25.135605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b99e8-91b1-4b79-84e6-791b20a539fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:25.135605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b99e8-91b1-4b79-84e6-791b20a539fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:25.144811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b99e8-91b1-4b79-84e6-791b20a539fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:25.146295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b99e8-91b1-4b79-84e6-791b20a539fb service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:25.158308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1111/2242] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:56:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:56:25.187101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0608-3094-4ef1-9e17-35ea774f10f1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:25.209129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:56:25.254021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0608-3094-4ef1-9e17-35ea774f10f1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:25.259293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:25.260910 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:25.261436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:25.261721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] PATH_INFO: `/v3/users/a3970092c268469cae2a83d73d20f875` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:25.262959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3970092c268469cae2a83d73d20f875)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:25.264884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:25.472171 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3970092c268469cae2a83d73d20f875'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:25.473986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3970092c268469cae2a83d73d20f875'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:25.575421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3970092c268469cae2a83d73d20f875'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:25.584774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3970092c268469cae2a83d73d20f875'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:25.605387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3970092c268469cae2a83d73d20f875'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:25.612054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1132/2243] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:56:25 2022] DELETE /identity/v3/users/a3970092c268469cae2a83d73d20f875 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:56:25.616464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e05d3f-e6c4-4788-b779-e3ab5caa33ee None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:25.617647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e05d3f-e6c4-4788-b779-e3ab5caa33ee None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:25.618969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e05d3f-e6c4-4788-b779-e3ab5caa33ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:25.642122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-80e05d3f-e6c4-4788-b779-e3ab5caa33ee None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:56:25.643445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-80e05d3f-e6c4-4788-b779-e3ab5caa33ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:25.717396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1112/2244] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:56:25.734288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:25.735859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:25.795997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a861fc-6e43-4833-8ac7-d25e8e3ef430 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:25.800135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6997fc-eba9-4e1d-aa04-42329274d8aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:25.801236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6997fc-eba9-4e1d-aa04-42329274d8aa service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:25.801468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6997fc-eba9-4e1d-aa04-42329274d8aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:25.801737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6997fc-eba9-4e1d-aa04-42329274d8aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:25.816201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6997fc-eba9-4e1d-aa04-42329274d8aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:25.817670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6997fc-eba9-4e1d-aa04-42329274d8aa service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:25.829791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1133/2245] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:56:26.543372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e05d3f-e6c4-4788-b779-e3ab5caa33ee None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:56:26.545170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-80e05d3f-e6c4-4788-b779-e3ab5caa33ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:26.561526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e05d3f-e6c4-4788-b779-e3ab5caa33ee None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:56:26.567224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:56:26.567875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:26.568158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:26.568440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] PATH_INFO: `/v3/projects/bb2bd7e390bd444cb2f9d819d8a2ef6a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:26.569571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2bd7e390bd444cb2f9d819d8a2ef6a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:56:26.571540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:56:26.712284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2bd7e390bd444cb2f9d819d8a2ef6a', 'request_id': 'req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:26.780853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2bd7e390bd444cb2f9d819d8a2ef6a', 'request_id': 'req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:26.787991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2bd7e390bd444cb2f9d819d8a2ef6a', 'request_id': 'req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:26.801101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] The token cache is being invalidate because project bb2bd7e390bd444cb2f9d819d8a2ef6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:56:26.802603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:26.802603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2a5a38d8-b6d1-4f18-ac6a-741650eb6c28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:56:26.808833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1113/2246] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:56:26 2022] DELETE /identity/v3/projects/bb2bd7e390bd444cb2f9d819d8a2ef6a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:56:35.502325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febff674-9f13-4b91-8f32-f3220983985f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:35.502325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febff674-9f13-4b91-8f32-f3220983985f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:35.502325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febff674-9f13-4b91-8f32-f3220983985f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:35.521320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-febff674-9f13-4b91-8f32-f3220983985f None None] MFA Rules not processed for user `e5582d10ebef459084fe287c18568f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:56:35.522477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-febff674-9f13-4b91-8f32-f3220983985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:35.577691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:56:35.615598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1134/2247] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:56:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:56:36.797658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:56:45.785651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=7 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:56:46.748125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217874b-915b-4494-86b4-037da1fc2292 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:46.748667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217874b-915b-4494-86b4-037da1fc2292 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:46.748667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217874b-915b-4494-86b4-037da1fc2292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:46.770952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-5217874b-915b-4494-86b4-037da1fc2292 None None] MFA Rules not processed for user `87a0f4d8b8c341ed85618e70adb2def4`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:56:46.772213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5217874b-915b-4494-86b4-037da1fc2292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:46.829187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1114/2248] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:56:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:56:48.906483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02da9e-3c83-4a1f-9366-67d071bb79c4 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:56:48.907044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02da9e-3c83-4a1f-9366-67d071bb79c4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:56:48.907188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02da9e-3c83-4a1f-9366-67d071bb79c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:56:48.926564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6c02da9e-3c83-4a1f-9366-67d071bb79c4 None None] MFA Rules not processed for user `547bce0d8133462c95bcb6f272776a27`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:56:48.927773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6c02da9e-3c83-4a1f-9366-67d071bb79c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:56:48.984917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1135/2249] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:56:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:56:56.822565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=25 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:56:58.976237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=25 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:02.230211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217874b-915b-4494-86b4-037da1fc2292 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:02.232008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5217874b-915b-4494-86b4-037da1fc2292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:02.289840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217874b-915b-4494-86b4-037da1fc2292 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:02.293415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:02.294027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:02.294313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:02.294689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:02.296434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:02.365811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:02.367424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:02.383525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1115/2250] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:03.572191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02da9e-3c83-4a1f-9366-67d071bb79c4 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:03.574865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6c02da9e-3c83-4a1f-9366-67d071bb79c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:03.588421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02da9e-3c83-4a1f-9366-67d071bb79c4 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:03.592024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d02ae-ad0f-4acd-8afd-387a5b6868e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:03.592640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d02ae-ad0f-4acd-8afd-387a5b6868e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:03.592927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d02ae-ad0f-4acd-8afd-387a5b6868e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:03.593203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d02ae-ad0f-4acd-8afd-387a5b6868e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:03.594953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-693d02ae-ad0f-4acd-8afd-387a5b6868e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:03.608848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d02ae-ad0f-4acd-8afd-387a5b6868e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:03.610227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d02ae-ad0f-4acd-8afd-387a5b6868e7 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:03.621890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1136/2251] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:12.377362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=54 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:13.616328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=6 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:22.136854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:22.139128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:22.155814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b249a83-35f2-401d-8200-258dc93f267d service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:22.161925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:22.163717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:22.163717 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:22.164571 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:22.166623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:22.239767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:22.241490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:22.251344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1116/2252] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:22.791051 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6736660-70d9-4cfe-99f1-46263c9171e5 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:22.791545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6736660-70d9-4cfe-99f1-46263c9171e5 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:22.791645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6736660-70d9-4cfe-99f1-46263c9171e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:22.812024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b6736660-70d9-4cfe-99f1-46263c9171e5 None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:57:22.813262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b6736660-70d9-4cfe-99f1-46263c9171e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:22.879531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1137/2253] 10.222.0.82 () {58 vars in 967 bytes} [Wed Aug 24 03:57:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:57:23.060680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:23.062148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:23.132721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d598b-cf2e-427e-9e89-98c49004758e service nova] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:23.138891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b7bf5-00ec-44a0-ac60-a865c43bb87b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:23.138891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b7bf5-00ec-44a0-ac60-a865c43bb87b service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:23.138891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b7bf5-00ec-44a0-ac60-a865c43bb87b service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:23.140279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b7bf5-00ec-44a0-ac60-a865c43bb87b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:23.147535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b7bf5-00ec-44a0-ac60-a865c43bb87b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:23.149288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b7bf5-00ec-44a0-ac60-a865c43bb87b service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:23.157785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1117/2254] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:57:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:57:26.665797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0882d-9d5d-4121-a671-993c610d5569 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:26.666331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0882d-9d5d-4121-a671-993c610d5569 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:26.666449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0882d-9d5d-4121-a671-993c610d5569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:26.688046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-7be0882d-9d5d-4121-a671-993c610d5569 None None] MFA Rules not processed for user `c7e29563153a4ecbbe7447bed0324df7`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:57:26.689473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7be0882d-9d5d-4121-a671-993c610d5569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:26.698126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1138/2255] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:57:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:57:27.224003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b7bf5-00ec-44a0-ac60-a865c43bb87b service swift] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:27.234563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b7bf5-00ec-44a0-ac60-a865c43bb87b service swift] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:27.238277 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f60bd0-2d5a-4bde-9e35-04e0af1bf531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:27.238926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f60bd0-2d5a-4bde-9e35-04e0af1bf531 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:27.239212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f60bd0-2d5a-4bde-9e35-04e0af1bf531 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:27.239490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f60bd0-2d5a-4bde-9e35-04e0af1bf531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:27.241850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-79f60bd0-2d5a-4bde-9e35-04e0af1bf531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:27.252467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f60bd0-2d5a-4bde-9e35-04e0af1bf531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:27.253926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f60bd0-2d5a-4bde-9e35-04e0af1bf531 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:27.264011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1118/2256] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:27.865464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0882d-9d5d-4121-a671-993c610d5569 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:27.867269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7be0882d-9d5d-4121-a671-993c610d5569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:27.972256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0882d-9d5d-4121-a671-993c610d5569 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:27.978253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7677a-4dad-492a-99ef-84e4232a5c81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:27.979025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7677a-4dad-492a-99ef-84e4232a5c81 service placement] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:27.979379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7677a-4dad-492a-99ef-84e4232a5c81 service placement] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:27.979719 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7677a-4dad-492a-99ef-84e4232a5c81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:27.981600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-73d7677a-4dad-492a-99ef-84e4232a5c81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:27.996667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7677a-4dad-492a-99ef-84e4232a5c81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:27.998478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7677a-4dad-492a-99ef-84e4232a5c81 service placement] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:28.012751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1139/2257] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:28.532438 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c9955-44c8-44a9-bf1e-0409f78499e1 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:28.532952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c9955-44c8-44a9-bf1e-0409f78499e1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:28.534247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c9955-44c8-44a9-bf1e-0409f78499e1 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:28.536899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1119/2258] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:57:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:57:28.543523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e03cb-3393-4115-a2ff-efcb4603f087 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:28.543735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e03cb-3393-4115-a2ff-efcb4603f087 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:28.544118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e03cb-3393-4115-a2ff-efcb4603f087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:28.566225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-234e03cb-3393-4115-a2ff-efcb4603f087 None None] MFA Rules not processed for user `cff23ae056ff452588cd0b6520e96962`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:57:28.567684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-234e03cb-3393-4115-a2ff-efcb4603f087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:28.578231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1140/2259] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:57:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:57:28.597012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c9955-44c8-44a9-bf1e-0409f78499e1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:28.598782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d32c9955-44c8-44a9-bf1e-0409f78499e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:28.618637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c9955-44c8-44a9-bf1e-0409f78499e1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:28.623642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111eedf5-420f-4532-bc6b-24570e6688ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:28.624243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111eedf5-420f-4532-bc6b-24570e6688ea service placement] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:28.624528 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111eedf5-420f-4532-bc6b-24570e6688ea service placement] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:28.624807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111eedf5-420f-4532-bc6b-24570e6688ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:28.635178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111eedf5-420f-4532-bc6b-24570e6688ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:28.636774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111eedf5-420f-4532-bc6b-24570e6688ea service placement] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:28.646613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1120/2260] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:32.153656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e03cb-3393-4115-a2ff-efcb4603f087 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:32.163106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e03cb-3393-4115-a2ff-efcb4603f087 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:32.166789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbfc0e-1a59-4fed-86c6-601c7c8deb2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:32.167444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbfc0e-1a59-4fed-86c6-601c7c8deb2e service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:32.167682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbfc0e-1a59-4fed-86c6-601c7c8deb2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:32.167950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbfc0e-1a59-4fed-86c6-601c7c8deb2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:32.169738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d0bbfc0e-1a59-4fed-86c6-601c7c8deb2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:32.181085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbfc0e-1a59-4fed-86c6-601c7c8deb2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:32.182790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbfc0e-1a59-4fed-86c6-601c7c8deb2e service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:32.192788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1141/2261] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:33.099006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111eedf5-420f-4532-bc6b-24570e6688ea service placement] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:33.102008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-111eedf5-420f-4532-bc6b-24570e6688ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:33.173699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111eedf5-420f-4532-bc6b-24570e6688ea service placement] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:33.181218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:33.181218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:33.181786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:33.182010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] PATH_INFO: `/v3/users/c662a5d21958400daa83733a93ca4c11` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:33.183536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=c662a5d21958400daa83733a93ca4c11)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:33.185303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:33.188078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:33.415247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c662a5d21958400daa83733a93ca4c11'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:33.536339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c662a5d21958400daa83733a93ca4c11'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:33.553502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c662a5d21958400daa83733a93ca4c11'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:33.563410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c662a5d21958400daa83733a93ca4c11'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:33.567417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c662a5d21958400daa83733a93ca4c11'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:33.570269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1121/2262] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:57:33 2022] DELETE /identity/v3/users/c662a5d21958400daa83733a93ca4c11 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:57:33.975249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbfc0e-1a59-4fed-86c6-601c7c8deb2e service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:33.992185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbfc0e-1a59-4fed-86c6-601c7c8deb2e service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:34.061970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08a86f-6ef1-48e3-a1ba-6b7dfd707298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:34.062624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08a86f-6ef1-48e3-a1ba-6b7dfd707298 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:34.062907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08a86f-6ef1-48e3-a1ba-6b7dfd707298 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:34.063185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08a86f-6ef1-48e3-a1ba-6b7dfd707298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:34.065026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ea08a86f-6ef1-48e3-a1ba-6b7dfd707298 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:34.070806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:34.076342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:34.077793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:34.155241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f2921-508e-4e3a-94e8-421f45d84bed admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:34.156523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08a86f-6ef1-48e3-a1ba-6b7dfd707298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:34.158223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08a86f-6ef1-48e3-a1ba-6b7dfd707298 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:34.160461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:34.161105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:34.161394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:34.161675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] PATH_INFO: `/v3/projects/5d84c2db19c24ef49d5012dd6c28cb7b` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:34.164121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d84c2db19c24ef49d5012dd6c28cb7b)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:34.166061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:34.168720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1142/2263] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:34.321458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d84c2db19c24ef49d5012dd6c28cb7b', 'request_id': 'req-b12e5007-bccc-4e87-95c9-b662ae2437ef'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.421527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d84c2db19c24ef49d5012dd6c28cb7b', 'request_id': 'req-b12e5007-bccc-4e87-95c9-b662ae2437ef'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.429182 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d84c2db19c24ef49d5012dd6c28cb7b', 'request_id': 'req-b12e5007-bccc-4e87-95c9-b662ae2437ef'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.440379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] The token cache is being invalidate because project 5d84c2db19c24ef49d5012dd6c28cb7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:57:34.440668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.441307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.442900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1122/2264] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:57:34 2022] DELETE /identity/v3/projects/5d84c2db19c24ef49d5012dd6c28cb7b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:57:34.447094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08a86f-6ef1-48e3-a1ba-6b7dfd707298 service nova] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:34.457778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08a86f-6ef1-48e3-a1ba-6b7dfd707298 service nova] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:34.520571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:34.522190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:34.522924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:34.523317 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] PATH_INFO: `/v3/users/5c2f361c4940443986148753ff4f27aa` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:34.525835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c2f361c4940443986148753ff4f27aa)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:34.529802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:34.773332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c2f361c4940443986148753ff4f27aa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.773913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c2f361c4940443986148753ff4f27aa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.890499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c2f361c4940443986148753ff4f27aa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.898997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c2f361c4940443986148753ff4f27aa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.911747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c2f361c4940443986148753ff4f27aa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:34.920622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1143/2265] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:57:34 2022] DELETE /identity/v3/users/5c2f361c4940443986148753ff4f27aa => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:57:35.223777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:35.225507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:35.239075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e5007-bccc-4e87-95c9-b662ae2437ef admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:35.247803 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:35.302416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:35.302416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:35.302416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:35.302416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] PATH_INFO: `/v3/projects/8e7cb7f74f9e4efcbfa816fe442475f8` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:35.303589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e7cb7f74f9e4efcbfa816fe442475f8)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:35.303738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:35.416301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e7cb7f74f9e4efcbfa816fe442475f8', 'request_id': 'req-2e09695b-399a-4883-9711-afff04522c02'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:35.622218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e7cb7f74f9e4efcbfa816fe442475f8', 'request_id': 'req-2e09695b-399a-4883-9711-afff04522c02'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:35.628955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e7cb7f74f9e4efcbfa816fe442475f8', 'request_id': 'req-2e09695b-399a-4883-9711-afff04522c02'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:35.640282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] The token cache is being invalidate because project 8e7cb7f74f9e4efcbfa816fe442475f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:57:35.640579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:35.641303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:57:35.643090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1123/2266] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:57:35 2022] DELETE /identity/v3/projects/8e7cb7f74f9e4efcbfa816fe442475f8 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:57:36.008437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:36.009865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:36.040056 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:36.071422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3c30d-7f65-4c13-9b81-f76a0401a000 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:36.122990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db761bd1-b83f-4b43-822f-32acd09388fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:36.123546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db761bd1-b83f-4b43-822f-32acd09388fa service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:36.123829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db761bd1-b83f-4b43-822f-32acd09388fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:36.124094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db761bd1-b83f-4b43-822f-32acd09388fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:36.125889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-db761bd1-b83f-4b43-822f-32acd09388fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:36.187580 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db761bd1-b83f-4b43-822f-32acd09388fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:36.189075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db761bd1-b83f-4b43-822f-32acd09388fa service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:36.200423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1144/2267] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:44.992806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:44.995087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:45.127159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09695b-399a-4883-9711-afff04522c02 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:45.137259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:45.138001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:45.138337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:45.138663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:45.140817 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:45.157972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:45.160337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:45.186028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1124/2268] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:45.809100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db761bd1-b83f-4b43-822f-32acd09388fa service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:45.835335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db761bd1-b83f-4b43-822f-32acd09388fa service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:45.844458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43223375-02f6-4b27-89fb-e92e8377fbba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:45.845451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43223375-02f6-4b27-89fb-e92e8377fbba service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:45.845742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43223375-02f6-4b27-89fb-e92e8377fbba service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:45.846023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43223375-02f6-4b27-89fb-e92e8377fbba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:45.851854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-43223375-02f6-4b27-89fb-e92e8377fbba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:45.872752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43223375-02f6-4b27-89fb-e92e8377fbba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:45.874559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43223375-02f6-4b27-89fb-e92e8377fbba service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:45.899674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1145/2269] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:49.330293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:49.331812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:49.346350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9235e-905c-47cd-b214-3bd2d7e9eea1 service nova] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:49.353895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:49.354906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:49.355196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:49.355473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:49.357265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:49.387241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:49.423035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:49.424702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:49.434674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1125/2270] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:55.883096 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=63 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:59.557716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=16 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:00.469244 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349d00-9467-429f-8621-f713e295daf0 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:00.469811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349d00-9467-429f-8621-f713e295daf0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:00.469811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349d00-9467-429f-8621-f713e295daf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:00.490475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-c0349d00-9467-429f-8621-f713e295daf0 None None] MFA Rules not processed for user `e5582d10ebef459084fe287c18568f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:00.491721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c0349d00-9467-429f-8621-f713e295daf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:00.548427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1146/2271] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:01.502866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:01.504325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:01.565079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913d525-eb6f-42d3-a5fd-eeef15a3db95 service nova] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:01.569093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285708-52ef-42a9-b8fc-078679404cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:01.569702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285708-52ef-42a9-b8fc-078679404cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:01.569984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285708-52ef-42a9-b8fc-078679404cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:01.570260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285708-52ef-42a9-b8fc-078679404cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:01.572081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6a285708-52ef-42a9-b8fc-078679404cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:01.637165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a285708-52ef-42a9-b8fc-078679404cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:01.638668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a285708-52ef-42a9-b8fc-078679404cc6 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:01.651661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1126/2272] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:06.345561 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0349d00-9467-429f-8621-f713e295daf0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:06.347291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c0349d00-9467-429f-8621-f713e295daf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:06.372040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0349d00-9467-429f-8621-f713e295daf0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:06.374690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:06.375484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:06.375867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:06.376240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] PATH_INFO: `/v3/users/547bce0d8133462c95bcb6f272776a27` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:06.377922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=547bce0d8133462c95bcb6f272776a27)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:06.380527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:06.521036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547bce0d8133462c95bcb6f272776a27'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:06.521894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547bce0d8133462c95bcb6f272776a27'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:06.614733 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547bce0d8133462c95bcb6f272776a27'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:06.622661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547bce0d8133462c95bcb6f272776a27'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:06.636637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c72c3a2e-85ba-444a-a72b-4d98570cebb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547bce0d8133462c95bcb6f272776a27'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:06.644000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1147/2273] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:58:06 2022] DELETE /identity/v3/users/547bce0d8133462c95bcb6f272776a27 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:58:06.958223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285708-52ef-42a9-b8fc-078679404cc6 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:06.969475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285708-52ef-42a9-b8fc-078679404cc6 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:07.041698 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:07.042344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.042663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.042976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] PATH_INFO: `/v3/projects/63408551e1b24a86b8f0059f1a948896` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:07.044139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=63408551e1b24a86b8f0059f1a948896)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:07.046118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:07.156128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63408551e1b24a86b8f0059f1a948896', 'request_id': 'req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:07.160013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:07.276524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63408551e1b24a86b8f0059f1a948896', 'request_id': 'req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:07.283587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63408551e1b24a86b8f0059f1a948896', 'request_id': 'req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:07.294044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] The token cache is being invalidate because project 63408551e1b24a86b8f0059f1a948896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:58:07.294339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:07.295010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c9377da9-d1cc-43ae-8ecc-58fdb71d46cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:07.296494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1127/2274] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:58:06 2022] DELETE /identity/v3/projects/63408551e1b24a86b8f0059f1a948896 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:58:07.302047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799a57c-9272-41b9-a840-b0aa6af117e4 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.302333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799a57c-9272-41b9-a840-b0aa6af117e4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.302642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799a57c-9272-41b9-a840-b0aa6af117e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:07.323573 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d799a57c-9272-41b9-a840-b0aa6af117e4 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:07.324822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d799a57c-9272-41b9-a840-b0aa6af117e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:07.390609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1148/2275] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:58:07.398456 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde6a96-2ecb-4a51-aa1e-6c860091c478 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.398777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde6a96-2ecb-4a51-aa1e-6c860091c478 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.399047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde6a96-2ecb-4a51-aa1e-6c860091c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:07.418610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6cde6a96-2ecb-4a51-aa1e-6c860091c478 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:07.419799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6cde6a96-2ecb-4a51-aa1e-6c860091c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:07.428806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1128/2276] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:07.433090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799a57c-9272-41b9-a840-b0aa6af117e4 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:07.444224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799a57c-9272-41b9-a840-b0aa6af117e4 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:07.449028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24ce1d-b035-4267-8922-de0b99ff5312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:07.449636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24ce1d-b035-4267-8922-de0b99ff5312 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.449917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24ce1d-b035-4267-8922-de0b99ff5312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.450192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24ce1d-b035-4267-8922-de0b99ff5312 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:07.450877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24ce1d-b035-4267-8922-de0b99ff5312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:58:07.451179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24ce1d-b035-4267-8922-de0b99ff5312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:07.452873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24ce1d-b035-4267-8922-de0b99ff5312 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:07.465568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1149/2277] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:58:07.471628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde6a96-2ecb-4a51-aa1e-6c860091c478 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:07.480716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde6a96-2ecb-4a51-aa1e-6c860091c478 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:07.487003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad93df-f078-4b6e-a00f-80ac9f65c941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:07.487003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad93df-f078-4b6e-a00f-80ac9f65c941 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.487257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad93df-f078-4b6e-a00f-80ac9f65c941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.487517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad93df-f078-4b6e-a00f-80ac9f65c941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:07.488152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad93df-f078-4b6e-a00f-80ac9f65c941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:07.490701 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad93df-f078-4b6e-a00f-80ac9f65c941 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:07.601222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1129/2278] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:58:07 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:58:07.605571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24ce1d-b035-4267-8922-de0b99ff5312 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:07.612516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:07.617266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24ce1d-b035-4267-8922-de0b99ff5312 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:07.681598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13952a6-f9e0-4655-a848-a3ea64708e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:07.682136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13952a6-f9e0-4655-a848-a3ea64708e36 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.682419 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13952a6-f9e0-4655-a848-a3ea64708e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.682729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13952a6-f9e0-4655-a848-a3ea64708e36 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:07.683360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13952a6-f9e0-4655-a848-a3ea64708e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:07.685330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13952a6-f9e0-4655-a848-a3ea64708e36 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:07.804311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1150/2279] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:58:07 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:58:07.808287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad93df-f078-4b6e-a00f-80ac9f65c941 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:07.818880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad93df-f078-4b6e-a00f-80ac9f65c941 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:07.824541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334dba2-c101-4204-8d70-ed1b9b6780e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:07.825155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334dba2-c101-4204-8d70-ed1b9b6780e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.825439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334dba2-c101-4204-8d70-ed1b9b6780e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.825716 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334dba2-c101-4204-8d70-ed1b9b6780e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:07.826363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334dba2-c101-4204-8d70-ed1b9b6780e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:07.828003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334dba2-c101-4204-8d70-ed1b9b6780e5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:07.839055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1130/2280] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:58:07.843130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13952a6-f9e0-4655-a848-a3ea64708e36 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:07.854271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13952a6-f9e0-4655-a848-a3ea64708e36 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:07.858984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77bcfd-4a40-4015-93f1-c8be7ad72b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:07.859618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77bcfd-4a40-4015-93f1-c8be7ad72b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.859902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77bcfd-4a40-4015-93f1-c8be7ad72b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.860173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77bcfd-4a40-4015-93f1-c8be7ad72b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:07.860825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77bcfd-4a40-4015-93f1-c8be7ad72b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:07.862422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77bcfd-4a40-4015-93f1-c8be7ad72b47 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:07.873689 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1151/2281] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:58:07.877927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334dba2-c101-4204-8d70-ed1b9b6780e5 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:07.888545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334dba2-c101-4204-8d70-ed1b9b6780e5 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:07.893096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8400ef-9d33-43f4-8e79-ff366fd4b197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:07.893744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8400ef-9d33-43f4-8e79-ff366fd4b197 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.894024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8400ef-9d33-43f4-8e79-ff366fd4b197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.894675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8400ef-9d33-43f4-8e79-ff366fd4b197 admin admin] PATH_INFO: `/v3/projects/4b1b41a3dc9442ad9f47d480d57fc972/users/f93377bef018419dac5865e02816039a/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:07.909362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8400ef-9d33-43f4-8e79-ff366fd4b197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b1b41a3dc9442ad9f47d480d57fc972, user_id=f93377bef018419dac5865e02816039a, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:07.912824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8400ef-9d33-43f4-8e79-ff366fd4b197 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:07.980376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1131/2282] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:58:07 2022] PUT /identity/v3/projects/4b1b41a3dc9442ad9f47d480d57fc972/users/f93377bef018419dac5865e02816039a/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:58:07.991935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8641dbb-8335-4788-823d-8bc7aa9bf49e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:07.992387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8641dbb-8335-4788-823d-8bc7aa9bf49e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:07.992732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8641dbb-8335-4788-823d-8bc7aa9bf49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:08.027233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-f8641dbb-8335-4788-823d-8bc7aa9bf49e None None] MFA Rules not processed for user `f93377bef018419dac5865e02816039a`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:08.027233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f8641dbb-8335-4788-823d-8bc7aa9bf49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:08.094496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1152/2283] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:08.107017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8400ef-9d33-43f4-8e79-ff366fd4b197 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:08.108723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3a8400ef-9d33-43f4-8e79-ff366fd4b197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:08.183401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8400ef-9d33-43f4-8e79-ff366fd4b197 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:08.188956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed3dbd-d3d6-4da4-8fa8-e5746feca108 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:08.189563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed3dbd-d3d6-4da4-8fa8-e5746feca108 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:08.189842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed3dbd-d3d6-4da4-8fa8-e5746feca108 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:08.190111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed3dbd-d3d6-4da4-8fa8-e5746feca108 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:08.198385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed3dbd-d3d6-4da4-8fa8-e5746feca108 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:08.200007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed3dbd-d3d6-4da4-8fa8-e5746feca108 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:08.209527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1132/2284] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:10.790857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8641dbb-8335-4788-823d-8bc7aa9bf49e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:10.825924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8641dbb-8335-4788-823d-8bc7aa9bf49e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:10.828458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:10.829357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:10.829764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:10.830165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:10.832185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:10.846620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:10.848514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:10.859770 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1153/2285] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:13.643046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbda6d6-8e9a-450d-9862-bf04529a108f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:13.644731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbda6d6-8e9a-450d-9862-bf04529a108f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:13.644731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbda6d6-8e9a-450d-9862-bf04529a108f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:13.665699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6cbda6d6-8e9a-450d-9862-bf04529a108f None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:13.666992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6cbda6d6-8e9a-450d-9862-bf04529a108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:13.730572 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1133/2286] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:58:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:13.743404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:13.751825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:13.812316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:13.818873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88a183-839d-4071-ba44-01f25805a8ba service nova] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:13.824815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c2691-d047-4acb-afe3-44709224d797 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:13.825438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c2691-d047-4acb-afe3-44709224d797 service swift] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:13.825727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c2691-d047-4acb-afe3-44709224d797 service swift] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:13.827935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c2691-d047-4acb-afe3-44709224d797 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:13.835048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c2691-d047-4acb-afe3-44709224d797 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:13.836659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c2691-d047-4acb-afe3-44709224d797 service swift] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:13.844764 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1154/2287] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:58:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:58:16.452025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbda6d6-8e9a-450d-9862-bf04529a108f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:16.466359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:16.563823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbda6d6-8e9a-450d-9862-bf04529a108f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:16.582632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:16.582632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:16.582632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:16.582632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] PATH_INFO: `/v3/users/f93377bef018419dac5865e02816039a` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:16.584190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93377bef018419dac5865e02816039a)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:16.588713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:17.086829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93377bef018419dac5865e02816039a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:17.179156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93377bef018419dac5865e02816039a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:17.197902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93377bef018419dac5865e02816039a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:17.206431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93377bef018419dac5865e02816039a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:17.212605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93377bef018419dac5865e02816039a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:17.215806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1134/2288] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:58:16 2022] DELETE /identity/v3/users/f93377bef018419dac5865e02816039a => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:58:17.220121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894624f-8f47-4141-9793-7f5ee6ed4408 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:17.220635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894624f-8f47-4141-9793-7f5ee6ed4408 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:17.220899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894624f-8f47-4141-9793-7f5ee6ed4408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:17.262913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a894624f-8f47-4141-9793-7f5ee6ed4408 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:17.264179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a894624f-8f47-4141-9793-7f5ee6ed4408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:17.382160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1155/2289] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:17.396887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:17.398992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:17.497388 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c02f5-5763-46f4-b597-3ca3fc74db31 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:17.501357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332857a-c434-404f-8b06-40434a31d60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:17.502147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332857a-c434-404f-8b06-40434a31d60a service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:17.502439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332857a-c434-404f-8b06-40434a31d60a service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:17.502750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332857a-c434-404f-8b06-40434a31d60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:17.519062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332857a-c434-404f-8b06-40434a31d60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:17.521902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332857a-c434-404f-8b06-40434a31d60a service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:17.547804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1135/2290] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:18.098810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894624f-8f47-4141-9793-7f5ee6ed4408 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:18.098810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a894624f-8f47-4141-9793-7f5ee6ed4408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:18.127705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894624f-8f47-4141-9793-7f5ee6ed4408 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:18.133726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:18.134405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:18.134746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:18.135033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] PATH_INFO: `/v3/projects/4b1b41a3dc9442ad9f47d480d57fc972` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:18.136612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b1b41a3dc9442ad9f47d480d57fc972)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:18.139144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:18.297278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b1b41a3dc9442ad9f47d480d57fc972', 'request_id': 'req-9ccfec34-a37b-4a55-abec-c5decd74b996'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:18.315747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b1b41a3dc9442ad9f47d480d57fc972', 'request_id': 'req-9ccfec34-a37b-4a55-abec-c5decd74b996'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:18.407830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b1b41a3dc9442ad9f47d480d57fc972', 'request_id': 'req-9ccfec34-a37b-4a55-abec-c5decd74b996'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:18.434423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] The token cache is being invalidate because project 4b1b41a3dc9442ad9f47d480d57fc972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:58:18.435241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:18.436633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9ccfec34-a37b-4a55-abec-c5decd74b996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:18.438424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1156/2291] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:58:18 2022] DELETE /identity/v3/projects/4b1b41a3dc9442ad9f47d480d57fc972 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:58:18.444970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b022356-6853-4008-83dc-04fc8c745677 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:18.445213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b022356-6853-4008-83dc-04fc8c745677 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:18.445476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b022356-6853-4008-83dc-04fc8c745677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:18.470650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2b022356-6853-4008-83dc-04fc8c745677 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:18.472279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2b022356-6853-4008-83dc-04fc8c745677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:18.546583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:18.630699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1136/2292] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:18.635952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868f2c6-5fa1-45c5-b2c7-5b423563192f None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:18.636283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868f2c6-5fa1-45c5-b2c7-5b423563192f None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:18.636519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868f2c6-5fa1-45c5-b2c7-5b423563192f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:18.672494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-1868f2c6-5fa1-45c5-b2c7-5b423563192f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:18.677849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1868f2c6-5fa1-45c5-b2c7-5b423563192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:18.710272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1157/2293] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:18.711976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b022356-6853-4008-83dc-04fc8c745677 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:18.725495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b022356-6853-4008-83dc-04fc8c745677 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:18.731514 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005ed4e-c784-4250-b1a1-ad5cb0d28272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:18.732290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005ed4e-c784-4250-b1a1-ad5cb0d28272 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:18.732584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005ed4e-c784-4250-b1a1-ad5cb0d28272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:18.734474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005ed4e-c784-4250-b1a1-ad5cb0d28272 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:18.734474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005ed4e-c784-4250-b1a1-ad5cb0d28272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:58:18.734474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005ed4e-c784-4250-b1a1-ad5cb0d28272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:18.735821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005ed4e-c784-4250-b1a1-ad5cb0d28272 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:18.765437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1137/2294] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:58:18.767093 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868f2c6-5fa1-45c5-b2c7-5b423563192f None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:18.783395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868f2c6-5fa1-45c5-b2c7-5b423563192f None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:18.803330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103d0f7-48d3-4b59-935e-74e116001412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:18.804055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103d0f7-48d3-4b59-935e-74e116001412 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:18.804262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103d0f7-48d3-4b59-935e-74e116001412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:18.804512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103d0f7-48d3-4b59-935e-74e116001412 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:18.805155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103d0f7-48d3-4b59-935e-74e116001412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:18.807128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103d0f7-48d3-4b59-935e-74e116001412 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:18.921756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1158/2295] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:58:18 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:58:18.928116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005ed4e-c784-4250-b1a1-ad5cb0d28272 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:18.947533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005ed4e-c784-4250-b1a1-ad5cb0d28272 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:19.045365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941604c7-42d7-4bf5-b8ed-389ff2414e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:19.046442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941604c7-42d7-4bf5-b8ed-389ff2414e6c admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:19.047010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941604c7-42d7-4bf5-b8ed-389ff2414e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:19.047523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941604c7-42d7-4bf5-b8ed-389ff2414e6c admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:19.048406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941604c7-42d7-4bf5-b8ed-389ff2414e6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:19.052171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941604c7-42d7-4bf5-b8ed-389ff2414e6c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:19.153362 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1138/2296] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:58:18 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:58:19.157795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103d0f7-48d3-4b59-935e-74e116001412 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:19.170066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103d0f7-48d3-4b59-935e-74e116001412 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:19.177608 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8674ce9-40a1-4975-83d6-2e07163ad34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:19.178240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8674ce9-40a1-4975-83d6-2e07163ad34e admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:19.178537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8674ce9-40a1-4975-83d6-2e07163ad34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:19.178819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8674ce9-40a1-4975-83d6-2e07163ad34e admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:19.182900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8674ce9-40a1-4975-83d6-2e07163ad34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:19.186335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8674ce9-40a1-4975-83d6-2e07163ad34e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:19.199385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1159/2297] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:58:19.203318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941604c7-42d7-4bf5-b8ed-389ff2414e6c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:19.214811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941604c7-42d7-4bf5-b8ed-389ff2414e6c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:19.223792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611109a2-39bb-406f-ba84-0f9f740ea64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:19.224561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611109a2-39bb-406f-ba84-0f9f740ea64b admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:19.224847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611109a2-39bb-406f-ba84-0f9f740ea64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:19.225121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611109a2-39bb-406f-ba84-0f9f740ea64b admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:19.225837 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611109a2-39bb-406f-ba84-0f9f740ea64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:19.227541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611109a2-39bb-406f-ba84-0f9f740ea64b admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:19.243032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1139/2298] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:58:19.246808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8674ce9-40a1-4975-83d6-2e07163ad34e admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:19.268981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8674ce9-40a1-4975-83d6-2e07163ad34e admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:19.277775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c0628-319b-4cd3-9100-a4eb91a0a7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:19.278979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0628-319b-4cd3-9100-a4eb91a0a7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:19.279262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0628-319b-4cd3-9100-a4eb91a0a7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:19.279540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0628-319b-4cd3-9100-a4eb91a0a7d2 admin admin] PATH_INFO: `/v3/projects/b40e410826c24a06868cba810cb84fd4/users/aa9f88ca02504cc4a494227c4f568ad7/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:19.299202 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c0628-319b-4cd3-9100-a4eb91a0a7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40e410826c24a06868cba810cb84fd4, user_id=aa9f88ca02504cc4a494227c4f568ad7, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:19.306398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c0628-319b-4cd3-9100-a4eb91a0a7d2 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:19.397002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1160/2299] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:58:19 2022] PUT /identity/v3/projects/b40e410826c24a06868cba810cb84fd4/users/aa9f88ca02504cc4a494227c4f568ad7/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:58:19.408513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c993df-e2ba-4850-90a8-e17f9466152f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:19.408820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c993df-e2ba-4850-90a8-e17f9466152f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:19.409264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c993df-e2ba-4850-90a8-e17f9466152f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:19.456352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-27c993df-e2ba-4850-90a8-e17f9466152f None None] MFA Rules not processed for user `aa9f88ca02504cc4a494227c4f568ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:19.459022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-27c993df-e2ba-4850-90a8-e17f9466152f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:19.572394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1140/2300] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:19.585071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c0628-319b-4cd3-9100-a4eb91a0a7d2 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:19.587566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3c8c0628-319b-4cd3-9100-a4eb91a0a7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:19.606378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:19.769799 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c0628-319b-4cd3-9100-a4eb91a0a7d2 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:19.778760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b95bfc-27bf-4039-bb36-1ecd28f9a6e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:19.780061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b95bfc-27bf-4039-bb36-1ecd28f9a6e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:19.780361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b95bfc-27bf-4039-bb36-1ecd28f9a6e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:19.781565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b95bfc-27bf-4039-bb36-1ecd28f9a6e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:19.804914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b95bfc-27bf-4039-bb36-1ecd28f9a6e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:19.804914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b95bfc-27bf-4039-bb36-1ecd28f9a6e1 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:19.821982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1161/2301] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:29.563674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:29.814154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=22 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:30.926430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c993df-e2ba-4850-90a8-e17f9466152f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:30.994410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c993df-e2ba-4850-90a8-e17f9466152f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:30.999043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:31.000203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:31.000482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:31.000885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] PATH_INFO: `/v3/users/aa9f88ca02504cc4a494227c4f568ad7` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:31.002109 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9f88ca02504cc4a494227c4f568ad7)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:31.005007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:31.215144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9f88ca02504cc4a494227c4f568ad7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:31.361543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9f88ca02504cc4a494227c4f568ad7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:31.374986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9f88ca02504cc4a494227c4f568ad7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:31.383075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9f88ca02504cc4a494227c4f568ad7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:31.383774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9f88ca02504cc4a494227c4f568ad7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:31.389784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1141/2302] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:58:30 2022] DELETE /identity/v3/users/aa9f88ca02504cc4a494227c4f568ad7 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:58:31.395026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141ed5-255b-4ec3-8932-effe46843aea None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:31.395551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141ed5-255b-4ec3-8932-effe46843aea None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:31.395808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141ed5-255b-4ec3-8932-effe46843aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:31.415265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e9141ed5-255b-4ec3-8932-effe46843aea None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:31.416483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e9141ed5-255b-4ec3-8932-effe46843aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:31.476838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1162/2303] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:58:31.487063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:31.489498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:31.550191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ea4c3-baab-4041-82a6-3832efb12c87 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:31.554036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8bbc7-3c0c-40c6-9451-8853e8ad322b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:31.554675 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8bbc7-3c0c-40c6-9451-8853e8ad322b service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:31.554959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8bbc7-3c0c-40c6-9451-8853e8ad322b service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:31.555234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8bbc7-3c0c-40c6-9451-8853e8ad322b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:31.564303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8bbc7-3c0c-40c6-9451-8853e8ad322b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:31.566335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8bbc7-3c0c-40c6-9451-8853e8ad322b service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:31.576534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1142/2304] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:32.048943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9141ed5-255b-4ec3-8932-effe46843aea None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:32.050944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e9141ed5-255b-4ec3-8932-effe46843aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:32.063379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9141ed5-255b-4ec3-8932-effe46843aea None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:32.068697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:32.070250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.070250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.070930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] PATH_INFO: `/v3/projects/b40e410826c24a06868cba810cb84fd4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.072390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b40e410826c24a06868cba810cb84fd4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:32.075379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:32.249134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b40e410826c24a06868cba810cb84fd4', 'request_id': 'req-b6d81396-e971-4ce3-a2b6-d292cc222378'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:32.258290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b40e410826c24a06868cba810cb84fd4', 'request_id': 'req-b6d81396-e971-4ce3-a2b6-d292cc222378'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:32.338680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b40e410826c24a06868cba810cb84fd4', 'request_id': 'req-b6d81396-e971-4ce3-a2b6-d292cc222378'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:32.353055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] The token cache is being invalidate because project b40e410826c24a06868cba810cb84fd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:58:32.353350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:32.354040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b6d81396-e971-4ce3-a2b6-d292cc222378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:32.355582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1163/2305] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:58:32 2022] DELETE /identity/v3/projects/b40e410826c24a06868cba810cb84fd4 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:58:32.360541 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468aaef6-e226-4c42-aba3-a2dbcfb5ca7a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.361023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468aaef6-e226-4c42-aba3-a2dbcfb5ca7a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.361119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468aaef6-e226-4c42-aba3-a2dbcfb5ca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.382487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea41a1-392a-4426-be23-f207e345cc96 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.382487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea41a1-392a-4426-be23-f207e345cc96 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.383190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea41a1-392a-4426-be23-f207e345cc96 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.384358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1164/2306] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:58:32.384844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-468aaef6-e226-4c42-aba3-a2dbcfb5ca7a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:32.386981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-468aaef6-e226-4c42-aba3-a2dbcfb5ca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:32.390234 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69a51c-366e-46aa-8e9d-be54733b99fd None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.390668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69a51c-366e-46aa-8e9d-be54733b99fd None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.390968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69a51c-366e-46aa-8e9d-be54733b99fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.413226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-2a69a51c-366e-46aa-8e9d-be54733b99fd None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:32.414248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2a69a51c-366e-46aa-8e9d-be54733b99fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:32.422436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:32.490928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1143/2307] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:58:32.494106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1165/2308] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:58:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:32.500038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693bf5c-2938-4d67-8c0b-74eea51401fb None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.500328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693bf5c-2938-4d67-8c0b-74eea51401fb None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.504011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693bf5c-2938-4d67-8c0b-74eea51401fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.506151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69a51c-366e-46aa-8e9d-be54733b99fd None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:32.507974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2a69a51c-366e-46aa-8e9d-be54733b99fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:32.522316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-9693bf5c-2938-4d67-8c0b-74eea51401fb None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:32.523701 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9693bf5c-2938-4d67-8c0b-74eea51401fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:32.527201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:32.532379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1144/2309] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:58:32.536672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693bf5c-2938-4d67-8c0b-74eea51401fb None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:32.545779 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693bf5c-2938-4d67-8c0b-74eea51401fb None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:32.550799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9e341-5dec-4294-b965-880723cb7f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:32.551400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e341-5dec-4294-b965-880723cb7f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.551680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e341-5dec-4294-b965-880723cb7f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.551956 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9e341-5dec-4294-b965-880723cb7f20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.552885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e341-5dec-4294-b965-880723cb7f20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:58:32.553003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e341-5dec-4294-b965-880723cb7f20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:32.554672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9e341-5dec-4294-b965-880723cb7f20 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:32.567979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1145/2310] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:58:32.568625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69a51c-366e-46aa-8e9d-be54733b99fd None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:32.572393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f22e3-6d89-4c68-afe8-7e5a707a87ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ff3f53ddbb04eb8916b9ccfdb03dd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:32.575884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f22e3-6d89-4c68-afe8-7e5a707a87ed service glance] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.575884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f22e3-6d89-4c68-afe8-7e5a707a87ed service glance] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.575884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f22e3-6d89-4c68-afe8-7e5a707a87ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.578640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9e341-5dec-4294-b965-880723cb7f20 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:32.585371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f22e3-6d89-4c68-afe8-7e5a707a87ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:32.586875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f22e3-6d89-4c68-afe8-7e5a707a87ed service glance] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:32.589400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9e341-5dec-4294-b965-880723cb7f20 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:32.594209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d4fff-cb7e-4370-98c3-151a111d6189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:32.594832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d4fff-cb7e-4370-98c3-151a111d6189 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.595205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d4fff-cb7e-4370-98c3-151a111d6189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.595485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d4fff-cb7e-4370-98c3-151a111d6189 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.596159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d4fff-cb7e-4370-98c3-151a111d6189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:32.598098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d4fff-cb7e-4370-98c3-151a111d6189 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:32.598652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1166/2311] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:58:32.751641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1146/2312] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:58:32 2022] POST /identity/v3/projects => generated 361 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:58:32.757046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f22e3-6d89-4c68-afe8-7e5a707a87ed service glance] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:32.767562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f22e3-6d89-4c68-afe8-7e5a707a87ed service glance] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:32.834824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559ae0b-b4d0-4865-b921-6b2ba8861a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:32.835362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559ae0b-b4d0-4865-b921-6b2ba8861a50 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.835654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559ae0b-b4d0-4865-b921-6b2ba8861a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.835931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559ae0b-b4d0-4865-b921-6b2ba8861a50 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.836574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559ae0b-b4d0-4865-b921-6b2ba8861a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:32.838566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559ae0b-b4d0-4865-b921-6b2ba8861a50 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:32.871787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40c530-2802-44ab-ac06-3586ec63c626 None None] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.872283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40c530-2802-44ab-ac06-3586ec63c626 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.872701 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40c530-2802-44ab-ac06-3586ec63c626 None None] PATH_INFO: `/` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.874548 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1147/2313] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:58:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:58:32.879989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64517eeb-9cae-4031-adca-d6219cf17f65 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.880276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64517eeb-9cae-4031-adca-d6219cf17f65 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.880558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64517eeb-9cae-4031-adca-d6219cf17f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.901594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-64517eeb-9cae-4031-adca-d6219cf17f65 None None] MFA Rules not processed for user `f81cc77e85f84d358471dd9efab45cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:32.902872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-64517eeb-9cae-4031-adca-d6219cf17f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:32.953980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1167/2314] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:58:32 2022] POST /identity/v3/users => generated 281 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:58:32.955136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559ae0b-b4d0-4865-b921-6b2ba8861a50 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:32.955517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1148/2315] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:58:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:58:32.966319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559ae0b-b4d0-4865-b921-6b2ba8861a50 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:32.967393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64517eeb-9cae-4031-adca-d6219cf17f65 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:32.968736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-64517eeb-9cae-4031-adca-d6219cf17f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:32.971015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53949bde-b31a-4a34-a509-0af8c2319efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:32.972273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53949bde-b31a-4a34-a509-0af8c2319efd admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.972787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53949bde-b31a-4a34-a509-0af8c2319efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.972917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53949bde-b31a-4a34-a509-0af8c2319efd admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.973549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53949bde-b31a-4a34-a509-0af8c2319efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:32.975186 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53949bde-b31a-4a34-a509-0af8c2319efd admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:32.980863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64517eeb-9cae-4031-adca-d6219cf17f65 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:32.984179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9ff6e-28a6-41b3-a75e-8f2910d04458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:32.984777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9ff6e-28a6-41b3-a75e-8f2910d04458 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:32.985072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9ff6e-28a6-41b3-a75e-8f2910d04458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:32.985331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9ff6e-28a6-41b3-a75e-8f2910d04458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:32.985472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1168/2316] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:58:32.990026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53949bde-b31a-4a34-a509-0af8c2319efd admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:32.993901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9ff6e-28a6-41b3-a75e-8f2910d04458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:32.995380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9ff6e-28a6-41b3-a75e-8f2910d04458 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:33.004991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53949bde-b31a-4a34-a509-0af8c2319efd admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:33.005801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e3894-c7ed-40b7-a31b-758546c33aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:33.006038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e3894-c7ed-40b7-a31b-758546c33aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:33.006325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e3894-c7ed-40b7-a31b-758546c33aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:33.006634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e3894-c7ed-40b7-a31b-758546c33aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:33.007285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e3894-c7ed-40b7-a31b-758546c33aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:33.008887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e3894-c7ed-40b7-a31b-758546c33aaa admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:33.009688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1149/2317] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:58:33.020774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1169/2318] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:58:33.024761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9ff6e-28a6-41b3-a75e-8f2910d04458 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:33.036638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9ff6e-28a6-41b3-a75e-8f2910d04458 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:33.041425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a92c7d-bba7-4dda-826d-b28320395f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:33.042088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a92c7d-bba7-4dda-826d-b28320395f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:33.042378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a92c7d-bba7-4dda-826d-b28320395f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:33.042693 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a92c7d-bba7-4dda-826d-b28320395f3e admin admin] PATH_INFO: `/v3/projects/b21433261aca4d37aebf50d96a7aff85/users/1575c297fe3347acadfc2a6b816bcfbb/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:33.058409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a92c7d-bba7-4dda-826d-b28320395f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21433261aca4d37aebf50d96a7aff85, user_id=1575c297fe3347acadfc2a6b816bcfbb, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:33.061134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a92c7d-bba7-4dda-826d-b28320395f3e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:33.097633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881910f-c1eb-4ea3-a97e-ebe82adbe165 None None] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:33.097633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881910f-c1eb-4ea3-a97e-ebe82adbe165 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:33.097992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881910f-c1eb-4ea3-a97e-ebe82adbe165 None None] PATH_INFO: `/` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:33.099270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1170/2319] 10.222.0.82 () {54 vars in 921 bytes} [Wed Aug 24 03:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:58:33.104400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f975397-5150-4dd1-a412-169ffe10d6c3 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:33.104682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f975397-5150-4dd1-a412-169ffe10d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:33.104953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f975397-5150-4dd1-a412-169ffe10d6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:33.127808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-2f975397-5150-4dd1-a412-169ffe10d6c3 None None] MFA Rules not processed for user `0f34fe3fe912498b90dbf950e7fc999d`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:33.129023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2f975397-5150-4dd1-a412-169ffe10d6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:33.145812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1150/2320] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:58:33 2022] PUT /identity/v3/projects/b21433261aca4d37aebf50d96a7aff85/users/1575c297fe3347acadfc2a6b816bcfbb/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:58:33.156436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bafa3-54af-4104-9608-335a6c9fc3e4 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:33.156731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bafa3-54af-4104-9608-335a6c9fc3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:33.157016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bafa3-54af-4104-9608-335a6c9fc3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:33.188612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e20bafa3-54af-4104-9608-335a6c9fc3e4 None None] MFA Rules not processed for user `1575c297fe3347acadfc2a6b816bcfbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:33.190054 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e20bafa3-54af-4104-9608-335a6c9fc3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:33.209826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1171/2321] 10.222.0.82 () {58 vars in 1018 bytes} [Wed Aug 24 03:58:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:58:33.220406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f975397-5150-4dd1-a412-169ffe10d6c3 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:33.230088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f975397-5150-4dd1-a412-169ffe10d6c3 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:33.233899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:33.244442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:33.258761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1151/2322] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:58:33.289326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965a453-210a-480c-b666-b2e0b9e41bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:33.289984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965a453-210a-480c-b666-b2e0b9e41bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:33.290265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965a453-210a-480c-b666-b2e0b9e41bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:33.290575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965a453-210a-480c-b666-b2e0b9e41bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:33.296344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bafa3-54af-4104-9608-335a6c9fc3e4 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:33.297999 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e20bafa3-54af-4104-9608-335a6c9fc3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:33.358251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bafa3-54af-4104-9608-335a6c9fc3e4 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:33.360425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965a453-210a-480c-b666-b2e0b9e41bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:33.361870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c6a9e-1c1e-426d-8792-e70af4e5b6bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:33.362888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965a453-210a-480c-b666-b2e0b9e41bf1 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:33.363401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6a9e-1c1e-426d-8792-e70af4e5b6bc service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:33.363653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6a9e-1c1e-426d-8792-e70af4e5b6bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:33.363928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6a9e-1c1e-426d-8792-e70af4e5b6bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:33.372545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c6a9e-1c1e-426d-8792-e70af4e5b6bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:33.373359 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1172/2323] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:58:33.374075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c6a9e-1c1e-426d-8792-e70af4e5b6bc service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:33.384584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1152/2324] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:58:33.399997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965a453-210a-480c-b666-b2e0b9e41bf1 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:33.465060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965a453-210a-480c-b666-b2e0b9e41bf1 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:33.469857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:33.470493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:33.470804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:33.471074 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] PATH_INFO: `/v3/users/1575c297fe3347acadfc2a6b816bcfbb` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:33.472366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1575c297fe3347acadfc2a6b816bcfbb)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:33.474367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:33.725472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1575c297fe3347acadfc2a6b816bcfbb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:33.726135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1575c297fe3347acadfc2a6b816bcfbb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:33.852937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1575c297fe3347acadfc2a6b816bcfbb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:33.859070 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:33.861048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1575c297fe3347acadfc2a6b816bcfbb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:33.873530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1575c297fe3347acadfc2a6b816bcfbb'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:33.881026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1173/2325] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:58:33 2022] DELETE /identity/v3/users/1575c297fe3347acadfc2a6b816bcfbb => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:58:33.886190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7aafc-5d94-4010-a334-29c0588298d3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:33.886477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7aafc-5d94-4010-a334-29c0588298d3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:33.886785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7aafc-5d94-4010-a334-29c0588298d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:33.908198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-32d7aafc-5d94-4010-a334-29c0588298d3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:33.910256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-32d7aafc-5d94-4010-a334-29c0588298d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:33.972926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1153/2326] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:33.983167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:33.984656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:34.043447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9616b6-ecd2-4fc3-8b42-7f8c6a12eedd admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:34.047078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6ddc7-ca9e-4205-893c-8eb43b0b5619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:34.047684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6ddc7-ca9e-4205-893c-8eb43b0b5619 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:34.047966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6ddc7-ca9e-4205-893c-8eb43b0b5619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:34.048604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6ddc7-ca9e-4205-893c-8eb43b0b5619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:34.057645 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6ddc7-ca9e-4205-893c-8eb43b0b5619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:34.059120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6ddc7-ca9e-4205-893c-8eb43b0b5619 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:34.068247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1174/2327] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:34.454550 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7aafc-5d94-4010-a334-29c0588298d3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:34.457193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-32d7aafc-5d94-4010-a334-29c0588298d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:34.470691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7aafc-5d94-4010-a334-29c0588298d3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:34.477307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:34.478031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:34.478316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:34.478633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] PATH_INFO: `/v3/projects/b21433261aca4d37aebf50d96a7aff85` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:34.480138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21433261aca4d37aebf50d96a7aff85)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:34.481974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:34.695715 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21433261aca4d37aebf50d96a7aff85', 'request_id': 'req-a2f10926-b862-4409-a31c-22bef0af416a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:34.864374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21433261aca4d37aebf50d96a7aff85', 'request_id': 'req-a2f10926-b862-4409-a31c-22bef0af416a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:34.871311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21433261aca4d37aebf50d96a7aff85', 'request_id': 'req-a2f10926-b862-4409-a31c-22bef0af416a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:34.885340 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] The token cache is being invalidate because project b21433261aca4d37aebf50d96a7aff85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:58:34.885733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:34.886411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-a2f10926-b862-4409-a31c-22bef0af416a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:58:34.888232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1154/2328] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:58:34 2022] DELETE /identity/v3/projects/b21433261aca4d37aebf50d96a7aff85 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:58:34.895121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a6c44-cf88-49d2-8520-26a5f582618e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:34.896060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a6c44-cf88-49d2-8520-26a5f582618e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:34.896779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a6c44-cf88-49d2-8520-26a5f582618e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:34.917814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e38a6c44-cf88-49d2-8520-26a5f582618e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:34.919325 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e38a6c44-cf88-49d2-8520-26a5f582618e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:34.978737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1175/2329] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:34.986648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acd684-b4e9-44ed-9753-18c69e78697f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:34.993848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acd684-b4e9-44ed-9753-18c69e78697f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:34.993848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acd684-b4e9-44ed-9753-18c69e78697f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:35.008309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-76acd684-b4e9-44ed-9753-18c69e78697f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:35.009540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-76acd684-b4e9-44ed-9753-18c69e78697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:35.021732 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1155/2330] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:35.022728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a6c44-cf88-49d2-8520-26a5f582618e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:35.033033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a6c44-cf88-49d2-8520-26a5f582618e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:35.038568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15d01e-6244-4515-a7a4-9bf32cdf3fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:35.039193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15d01e-6244-4515-a7a4-9bf32cdf3fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:35.039477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15d01e-6244-4515-a7a4-9bf32cdf3fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:35.039826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15d01e-6244-4515-a7a4-9bf32cdf3fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:35.040476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15d01e-6244-4515-a7a4-9bf32cdf3fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:58:35.040771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15d01e-6244-4515-a7a4-9bf32cdf3fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:35.042385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15d01e-6244-4515-a7a4-9bf32cdf3fd9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:35.054556 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1176/2331] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:58:35.059309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acd684-b4e9-44ed-9753-18c69e78697f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:35.063937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:35.069309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acd684-b4e9-44ed-9753-18c69e78697f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:35.074075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766abce-da0e-4e06-b7e7-c5e61242bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:35.074735 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766abce-da0e-4e06-b7e7-c5e61242bd64 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:35.075017 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766abce-da0e-4e06-b7e7-c5e61242bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:35.075289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766abce-da0e-4e06-b7e7-c5e61242bd64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:35.075912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766abce-da0e-4e06-b7e7-c5e61242bd64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:35.077804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766abce-da0e-4e06-b7e7-c5e61242bd64 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:35.192074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1156/2332] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:58:35 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:58:35.196791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15d01e-6244-4515-a7a4-9bf32cdf3fd9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:35.208526 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15d01e-6244-4515-a7a4-9bf32cdf3fd9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:35.275394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505cbd-3940-429b-a9f2-8c74bf1c0f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:35.276044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505cbd-3940-429b-a9f2-8c74bf1c0f93 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:35.276351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505cbd-3940-429b-a9f2-8c74bf1c0f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:35.276657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505cbd-3940-429b-a9f2-8c74bf1c0f93 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:35.277313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78505cbd-3940-429b-a9f2-8c74bf1c0f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:35.279715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78505cbd-3940-429b-a9f2-8c74bf1c0f93 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:35.511371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:35.520460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1177/2333] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:58:35 2022] POST /identity/v3/users => generated 292 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:58:35.524906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766abce-da0e-4e06-b7e7-c5e61242bd64 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:35.532608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766abce-da0e-4e06-b7e7-c5e61242bd64 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:35.537844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffee66b-1e8a-4450-9bad-befb403c2b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:35.538446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffee66b-1e8a-4450-9bad-befb403c2b98 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:35.538760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffee66b-1e8a-4450-9bad-befb403c2b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:35.539039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffee66b-1e8a-4450-9bad-befb403c2b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:35.540007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffee66b-1e8a-4450-9bad-befb403c2b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:35.541393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffee66b-1e8a-4450-9bad-befb403c2b98 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:35.556503 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1157/2334] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:58:35.557911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505cbd-3940-429b-a9f2-8c74bf1c0f93 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:35.569715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505cbd-3940-429b-a9f2-8c74bf1c0f93 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:35.574407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80328259-0f24-431e-be02-dfbcb1edda27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:35.575126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80328259-0f24-431e-be02-dfbcb1edda27 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:35.575410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80328259-0f24-431e-be02-dfbcb1edda27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:35.575687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80328259-0f24-431e-be02-dfbcb1edda27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:35.576342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80328259-0f24-431e-be02-dfbcb1edda27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:35.577950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80328259-0f24-431e-be02-dfbcb1edda27 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:35.592678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1178/2335] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:58:35.593709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffee66b-1e8a-4450-9bad-befb403c2b98 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:35.603849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffee66b-1e8a-4450-9bad-befb403c2b98 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:35.608927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60051cf3-ad2d-4959-b2c5-f8f63a6a0e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:35.609594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60051cf3-ad2d-4959-b2c5-f8f63a6a0e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:35.609987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60051cf3-ad2d-4959-b2c5-f8f63a6a0e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:35.610142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60051cf3-ad2d-4959-b2c5-f8f63a6a0e5f admin admin] PATH_INFO: `/v3/projects/6e3d57c8abbd49e69c0caad57f7f9815/users/2206cb5005354ab4a716efb525c633e7/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:35.625447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60051cf3-ad2d-4959-b2c5-f8f63a6a0e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3d57c8abbd49e69c0caad57f7f9815, user_id=2206cb5005354ab4a716efb525c633e7, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:35.628893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60051cf3-ad2d-4959-b2c5-f8f63a6a0e5f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:35.710901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1158/2336] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:58:35 2022] PUT /identity/v3/projects/6e3d57c8abbd49e69c0caad57f7f9815/users/2206cb5005354ab4a716efb525c633e7/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:58:35.715992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf3f5b-7ca6-4597-a165-df82277b6875 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:35.726034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf3f5b-7ca6-4597-a165-df82277b6875 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:35.726034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf3f5b-7ca6-4597-a165-df82277b6875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:35.738117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9fbf3f5b-7ca6-4597-a165-df82277b6875 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:35.739376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9fbf3f5b-7ca6-4597-a165-df82277b6875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:35.798933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1179/2337] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:35.809440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60051cf3-ad2d-4959-b2c5-f8f63a6a0e5f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:35.811318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-60051cf3-ad2d-4959-b2c5-f8f63a6a0e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:35.872186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60051cf3-ad2d-4959-b2c5-f8f63a6a0e5f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:35.876446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda11c1-5b5f-4be8-bc56-d9416005d2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:35.877065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda11c1-5b5f-4be8-bc56-d9416005d2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:35.877376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda11c1-5b5f-4be8-bc56-d9416005d2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:35.877660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda11c1-5b5f-4be8-bc56-d9416005d2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:35.889468 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda11c1-5b5f-4be8-bc56-d9416005d2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:35.892539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda11c1-5b5f-4be8-bc56-d9416005d2e5 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:35.902175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1159/2338] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:36.583161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79804935-a198-4de8-b592-e02a6062e26e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:36.584177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79804935-a198-4de8-b592-e02a6062e26e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:36.585304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79804935-a198-4de8-b592-e02a6062e26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:36.635967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-79804935-a198-4de8-b592-e02a6062e26e None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:36.641193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-79804935-a198-4de8-b592-e02a6062e26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:36.698593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1180/2339] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 03:58:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:36.711562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda11c1-5b5f-4be8-bc56-d9416005d2e5 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:36.713117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3cda11c1-5b5f-4be8-bc56-d9416005d2e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:36.790278 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda11c1-5b5f-4be8-bc56-d9416005d2e5 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:36.795394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24221dd7-0fea-4472-840d-22c31a50d057 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:36.796014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24221dd7-0fea-4472-840d-22c31a50d057 service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:36.796371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24221dd7-0fea-4472-840d-22c31a50d057 service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:36.796577 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24221dd7-0fea-4472-840d-22c31a50d057 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:36.806448 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24221dd7-0fea-4472-840d-22c31a50d057 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:36.808135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24221dd7-0fea-4472-840d-22c31a50d057 service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:36.817434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1160/2340] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:58:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:58:37.491682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7666b8-a68d-4a0b-84c2-2b536546715c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:37.492075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7666b8-a68d-4a0b-84c2-2b536546715c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:37.492534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7666b8-a68d-4a0b-84c2-2b536546715c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:37.545304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-6b7666b8-a68d-4a0b-84c2-2b536546715c None None] MFA Rules not processed for user `2206cb5005354ab4a716efb525c633e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:37.547166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6b7666b8-a68d-4a0b-84c2-2b536546715c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:37.637231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1181/2341] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:37.649426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24221dd7-0fea-4472-840d-22c31a50d057 service swift] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:37.651768 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-24221dd7-0fea-4472-840d-22c31a50d057 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:37.710416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:37.728072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24221dd7-0fea-4472-840d-22c31a50d057 service swift] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:37.733317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a572c32-8719-4267-99ce-67526fb4dfa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:37.734399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a572c32-8719-4267-99ce-67526fb4dfa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:37.734912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a572c32-8719-4267-99ce-67526fb4dfa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:37.735214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a572c32-8719-4267-99ce-67526fb4dfa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:37.745804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a572c32-8719-4267-99ce-67526fb4dfa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:37.747543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a572c32-8719-4267-99ce-67526fb4dfa0 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:37.770679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1161/2342] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:39.673594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac668a-0cd8-41f8-ac07-0884d66deac7 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:39.674735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac668a-0cd8-41f8-ac07-0884d66deac7 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:39.675711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac668a-0cd8-41f8-ac07-0884d66deac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:39.717224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ffac668a-0cd8-41f8-ac07-0884d66deac7 None None] MFA Rules not processed for user `2206cb5005354ab4a716efb525c633e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:39.724320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ffac668a-0cd8-41f8-ac07-0884d66deac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:39.735432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1182/2343] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:58:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:40.014921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a572c32-8719-4267-99ce-67526fb4dfa0 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:40.016366 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6a572c32-8719-4267-99ce-67526fb4dfa0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:40.029969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a572c32-8719-4267-99ce-67526fb4dfa0 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:40.033575 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3d88e-2da0-4869-81d0-df7a2a885fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:40.034393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3d88e-2da0-4869-81d0-df7a2a885fdb service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:40.035319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3d88e-2da0-4869-81d0-df7a2a885fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:40.035601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3d88e-2da0-4869-81d0-df7a2a885fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:40.037368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c3d3d88e-2da0-4869-81d0-df7a2a885fdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:40.048988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3d88e-2da0-4869-81d0-df7a2a885fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:40.051634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3d88e-2da0-4869-81d0-df7a2a885fdb service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:40.060452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1162/2344] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:58:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:49.711239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=93 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:50.056704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=15 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:58:52.713396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405af055-2b19-4a26-9206-a199dbbdfdd9 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:52.713853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405af055-2b19-4a26-9206-a199dbbdfdd9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:52.715517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405af055-2b19-4a26-9206-a199dbbdfdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:52.734396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-405af055-2b19-4a26-9206-a199dbbdfdd9 None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:58:52.738540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-405af055-2b19-4a26-9206-a199dbbdfdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:58:52.744245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1183/2345] 10.222.0.82 () {58 vars in 967 bytes} [Wed Aug 24 03:58:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:58:52.757405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3d88e-2da0-4869-81d0-df7a2a885fdb service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:58:52.769625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3d88e-2da0-4869-81d0-df7a2a885fdb service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:58:52.774876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a8d17-439d-470d-aa4d-ebe2ee7c09fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:58:52.775511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a8d17-439d-470d-aa4d-ebe2ee7c09fb service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:58:52.775851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a8d17-439d-470d-aa4d-ebe2ee7c09fb service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:58:52.776355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a8d17-439d-470d-aa4d-ebe2ee7c09fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:58:52.790049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a8d17-439d-470d-aa4d-ebe2ee7c09fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:58:52.791748 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a8d17-439d-470d-aa4d-ebe2ee7c09fb service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:58:52.800811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1163/2346] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 03:58:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:59:02.729000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:02.796704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=6 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:21.375766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405af055-2b19-4a26-9206-a199dbbdfdd9 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:21.385420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405af055-2b19-4a26-9206-a199dbbdfdd9 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:21.388983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:21.389578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:21.389857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:21.390129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:21.392441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:21.404098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:21.405523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:21.415616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1184/2347] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:23.858444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a8d17-439d-470d-aa4d-ebe2ee7c09fb service swift] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:23.868808 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a8d17-439d-470d-aa4d-ebe2ee7c09fb service swift] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:23.872552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:23.873148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:23.873642 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:23.873926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:23.875906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:23.890599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:23.893119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:23.908957 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1164/2348] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:25.215046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:25.217298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:25.308723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0a540-3c49-4a71-ba7a-00d1174df78a service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:25.314479 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fe408-fcb5-4a42-b876-2a416efbfedf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:25.315142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fe408-fcb5-4a42-b876-2a416efbfedf service placement] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:25.315425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fe408-fcb5-4a42-b876-2a416efbfedf service placement] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:25.315700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fe408-fcb5-4a42-b876-2a416efbfedf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:25.317514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-083fe408-fcb5-4a42-b876-2a416efbfedf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:25.331403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fe408-fcb5-4a42-b876-2a416efbfedf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:25.332660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fe408-fcb5-4a42-b876-2a416efbfedf service placement] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:25.344282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1185/2349] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:25.375251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:25.377919 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:25.392554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef69d7-9034-4474-811c-4c2d77d114a0 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:25.398938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:25.399932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:25.400380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:25.400926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] PATH_INFO: `/v3/users/c7e29563153a4ecbbe7447bed0324df7` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:25.402296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e29563153a4ecbbe7447bed0324df7)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:25.404454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:25.631022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e29563153a4ecbbe7447bed0324df7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:25.730791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e29563153a4ecbbe7447bed0324df7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:25.746920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e29563153a4ecbbe7447bed0324df7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:25.757976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e29563153a4ecbbe7447bed0324df7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:25.757976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-59c3fc1a-740c-4a7a-b331-3c5970ba98d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e29563153a4ecbbe7447bed0324df7'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:25.763896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1165/2350] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:25 2022] DELETE /identity/v3/users/c7e29563153a4ecbbe7447bed0324df7 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:26.075440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fe408-fcb5-4a42-b876-2a416efbfedf service placement] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:26.097926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fe408-fcb5-4a42-b876-2a416efbfedf service placement] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:26.178067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:26.179016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:26.179300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:26.179576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] PATH_INFO: `/v3/projects/bdb802f52e1140d488ff0c6d739cd0d1` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:26.181066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb802f52e1140d488ff0c6d739cd0d1)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:26.185297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:26.343945 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb802f52e1140d488ff0c6d739cd0d1', 'request_id': 'req-4e8aae98-0525-4869-ac06-1fd8f74da94f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:26.358732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb802f52e1140d488ff0c6d739cd0d1', 'request_id': 'req-4e8aae98-0525-4869-ac06-1fd8f74da94f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:26.440592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb802f52e1140d488ff0c6d739cd0d1', 'request_id': 'req-4e8aae98-0525-4869-ac06-1fd8f74da94f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:26.453087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] The token cache is being invalidate because project bdb802f52e1140d488ff0c6d739cd0d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:26.453385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:26.455005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4e8aae98-0525-4869-ac06-1fd8f74da94f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:26.456971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1186/2351] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:26 2022] DELETE /identity/v3/projects/bdb802f52e1140d488ff0c6d739cd0d1 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:26.462172 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8440-9638-42f4-8745-c4679a296d26 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:26.462670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8440-9638-42f4-8745-c4679a296d26 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:26.462756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8440-9638-42f4-8745-c4679a296d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:26.487485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-90dd8440-9638-42f4-8745-c4679a296d26 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:26.489216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-90dd8440-9638-42f4-8745-c4679a296d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:26.558997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1166/2352] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:26.568920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ebcc9-d636-464a-8bed-04d8dcdbd726 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:26.569205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ebcc9-d636-464a-8bed-04d8dcdbd726 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:26.569670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ebcc9-d636-464a-8bed-04d8dcdbd726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:26.584907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:26.591236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-191ebcc9-d636-464a-8bed-04d8dcdbd726 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:26.592456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-191ebcc9-d636-464a-8bed-04d8dcdbd726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:26.601602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1187/2353] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:26.606043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd8440-9638-42f4-8745-c4679a296d26 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:26.622703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd8440-9638-42f4-8745-c4679a296d26 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:26.629430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fc6cf-1815-4319-86fb-3c1dd519222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:26.632436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fc6cf-1815-4319-86fb-3c1dd519222c admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:26.632436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fc6cf-1815-4319-86fb-3c1dd519222c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:26.632436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fc6cf-1815-4319-86fb-3c1dd519222c admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:26.632436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fc6cf-1815-4319-86fb-3c1dd519222c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:59:26.632436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fc6cf-1815-4319-86fb-3c1dd519222c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:26.633522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fc6cf-1815-4319-86fb-3c1dd519222c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:26.652469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1167/2354] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:59:26.657137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ebcc9-d636-464a-8bed-04d8dcdbd726 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:26.668150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ebcc9-d636-464a-8bed-04d8dcdbd726 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:26.676162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35785f50-7195-48f9-b9d9-40b8a6f8cb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:26.676439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35785f50-7195-48f9-b9d9-40b8a6f8cb6c admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:26.678238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35785f50-7195-48f9-b9d9-40b8a6f8cb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:26.678238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35785f50-7195-48f9-b9d9-40b8a6f8cb6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:26.678238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35785f50-7195-48f9-b9d9-40b8a6f8cb6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:26.685305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35785f50-7195-48f9-b9d9-40b8a6f8cb6c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:26.806771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1188/2355] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:26 2022] POST /identity/v3/projects => generated 405 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:26.810311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fc6cf-1815-4319-86fb-3c1dd519222c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:26.822007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fc6cf-1815-4319-86fb-3c1dd519222c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:26.878712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:26.890419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123d6ab-5e35-4753-bf27-1670b91d3b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:26.893938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123d6ab-5e35-4753-bf27-1670b91d3b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:26.893938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123d6ab-5e35-4753-bf27-1670b91d3b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:26.893938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123d6ab-5e35-4753-bf27-1670b91d3b54 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:26.893938 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123d6ab-5e35-4753-bf27-1670b91d3b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:26.894969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123d6ab-5e35-4753-bf27-1670b91d3b54 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:27.028856 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1168/2356] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:26 2022] POST /identity/v3/users => generated 303 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:27.033597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35785f50-7195-48f9-b9d9-40b8a6f8cb6c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:27.044800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35785f50-7195-48f9-b9d9-40b8a6f8cb6c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:27.050282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a091e4-c893-4dfd-a44a-aa15cbb2cdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:27.050876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a091e4-c893-4dfd-a44a-aa15cbb2cdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:27.051155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a091e4-c893-4dfd-a44a-aa15cbb2cdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:27.051426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a091e4-c893-4dfd-a44a-aa15cbb2cdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:27.052071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a091e4-c893-4dfd-a44a-aa15cbb2cdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:27.053673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a091e4-c893-4dfd-a44a-aa15cbb2cdb4 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:27.064177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1189/2357] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:27.068213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123d6ab-5e35-4753-bf27-1670b91d3b54 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:27.078461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123d6ab-5e35-4753-bf27-1670b91d3b54 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:27.083272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a6f91-dc71-41c7-82df-15211bed2787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:27.083898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6f91-dc71-41c7-82df-15211bed2787 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:27.084173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6f91-dc71-41c7-82df-15211bed2787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:27.084439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6f91-dc71-41c7-82df-15211bed2787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:27.085082 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a6f91-dc71-41c7-82df-15211bed2787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:27.086698 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a6f91-dc71-41c7-82df-15211bed2787 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:27.099661 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1169/2358] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:27.103893 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a091e4-c893-4dfd-a44a-aa15cbb2cdb4 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:27.116382 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a091e4-c893-4dfd-a44a-aa15cbb2cdb4 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:27.121252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dc222-485b-4b1d-8b36-11270a387264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:27.121937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dc222-485b-4b1d-8b36-11270a387264 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:27.122225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dc222-485b-4b1d-8b36-11270a387264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:27.122528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dc222-485b-4b1d-8b36-11270a387264 admin admin] PATH_INFO: `/v3/projects/480e1e85ebb84605920bff6e077ea7ce/users/bdee2ddc3b70463297d617faeb8f776c/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:27.141105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dc222-485b-4b1d-8b36-11270a387264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480e1e85ebb84605920bff6e077ea7ce, user_id=bdee2ddc3b70463297d617faeb8f776c, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:27.143982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dc222-485b-4b1d-8b36-11270a387264 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:27.241179 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1190/2359] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:27 2022] PUT /identity/v3/projects/480e1e85ebb84605920bff6e077ea7ce/users/bdee2ddc3b70463297d617faeb8f776c/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:27.246196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8d434-d726-4d29-a05a-1409c0a9356d None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:27.246489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8d434-d726-4d29-a05a-1409c0a9356d None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:27.246793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8d434-d726-4d29-a05a-1409c0a9356d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:27.268039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-dae8d434-d726-4d29-a05a-1409c0a9356d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:27.269557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dae8d434-d726-4d29-a05a-1409c0a9356d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:27.337308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1170/2360] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:27.347813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dc222-485b-4b1d-8b36-11270a387264 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:27.349610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4e4dc222-485b-4b1d-8b36-11270a387264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:27.417586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dc222-485b-4b1d-8b36-11270a387264 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:27.421433 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31e2f8-f274-46ed-b794-ddb1a7b56f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:27.422065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31e2f8-f274-46ed-b794-ddb1a7b56f3c service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:27.422350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31e2f8-f274-46ed-b794-ddb1a7b56f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:27.422807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31e2f8-f274-46ed-b794-ddb1a7b56f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:27.439888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31e2f8-f274-46ed-b794-ddb1a7b56f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:27.441347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31e2f8-f274-46ed-b794-ddb1a7b56f3c service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:27.451099 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1191/2361] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:28.299072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8d434-d726-4d29-a05a-1409c0a9356d None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:28.301253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dae8d434-d726-4d29-a05a-1409c0a9356d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:28.318284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8d434-d726-4d29-a05a-1409c0a9356d None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:28.322900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:28.323543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:28.323831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:28.324113 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] PATH_INFO: `/v3/users/e5582d10ebef459084fe287c18568f98` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:28.325249 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5582d10ebef459084fe287c18568f98)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:28.327248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:28.575181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5582d10ebef459084fe287c18568f98'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:28.730936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5582d10ebef459084fe287c18568f98'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:28.743601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5582d10ebef459084fe287c18568f98'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:28.750615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5582d10ebef459084fe287c18568f98'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:28.751785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5582d10ebef459084fe287c18568f98'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:28.757901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1171/2362] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:28 2022] DELETE /identity/v3/users/e5582d10ebef459084fe287c18568f98 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:28.980266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ac343-1dda-47e1-aee8-437cb43ff8a1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:28.980727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ac343-1dda-47e1-aee8-437cb43ff8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:28.980828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ac343-1dda-47e1-aee8-437cb43ff8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.022222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a00ac343-1dda-47e1-aee8-437cb43ff8a1 None None] MFA Rules not processed for user `bdee2ddc3b70463297d617faeb8f776c`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:29.023686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a00ac343-1dda-47e1-aee8-437cb43ff8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:29.058375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.060313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:29.118841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1192/2363] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:28 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:29.119626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:29.120548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ac343-1dda-47e1-aee8-437cb43ff8a1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.122032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a00ac343-1dda-47e1-aee8-437cb43ff8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:29.143477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ae91b-e59d-4ea6-839b-c91255a11bd8 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.148501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.149120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.149401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.149677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] PATH_INFO: `/v3/projects/46ef5b02d8da4d2eac26b6ccfb35e1a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.150991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ef5b02d8da4d2eac26b6ccfb35e1a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.153023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.169252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:29.196302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ac343-1dda-47e1-aee8-437cb43ff8a1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.200882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088155-0c6c-4e15-86f1-875848eabd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.201499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088155-0c6c-4e15-86f1-875848eabd5f admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.201784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088155-0c6c-4e15-86f1-875848eabd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.202062 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088155-0c6c-4e15-86f1-875848eabd5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.202746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37088155-0c6c-4e15-86f1-875848eabd5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.204808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37088155-0c6c-4e15-86f1-875848eabd5f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.270135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ef5b02d8da4d2eac26b6ccfb35e1a4', 'request_id': 'req-52db875d-f3a2-41b7-b6ed-3a8eae354580'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:29.305503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1193/2364] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:29 2022] POST /identity/v3/projects => generated 403 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:29.309846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088155-0c6c-4e15-86f1-875848eabd5f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.319525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ef5b02d8da4d2eac26b6ccfb35e1a4', 'request_id': 'req-52db875d-f3a2-41b7-b6ed-3a8eae354580'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:29.336738 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ef5b02d8da4d2eac26b6ccfb35e1a4', 'request_id': 'req-52db875d-f3a2-41b7-b6ed-3a8eae354580'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:29.353777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] The token cache is being invalidate because project 46ef5b02d8da4d2eac26b6ccfb35e1a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:29.354103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:29.354946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-52db875d-f3a2-41b7-b6ed-3a8eae354580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:29.361391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1172/2365] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:29 2022] DELETE /identity/v3/projects/46ef5b02d8da4d2eac26b6ccfb35e1a4 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:29.364597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6daae52-e13f-4da4-b47c-66d86a6777f7 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.364892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6daae52-e13f-4da4-b47c-66d86a6777f7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.369297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6daae52-e13f-4da4-b47c-66d86a6777f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.386812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088155-0c6c-4e15-86f1-875848eabd5f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.389681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e6daae52-e13f-4da4-b47c-66d86a6777f7 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:29.390937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e6daae52-e13f-4da4-b47c-66d86a6777f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:29.393159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0913e1c-7712-4b83-ae5c-f778cb43092d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.393776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0913e1c-7712-4b83-ae5c-f778cb43092d admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.394060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0913e1c-7712-4b83-ae5c-f778cb43092d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.394338 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0913e1c-7712-4b83-ae5c-f778cb43092d admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.395015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0913e1c-7712-4b83-ae5c-f778cb43092d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.396956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0913e1c-7712-4b83-ae5c-f778cb43092d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.399927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1173/2366] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:29.407385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce842f5-c312-4e68-a679-c5a5e604d6d6 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.407676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce842f5-c312-4e68-a679-c5a5e604d6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.407962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce842f5-c312-4e68-a679-c5a5e604d6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.427237 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-9ce842f5-c312-4e68-a679-c5a5e604d6d6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:29.428475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9ce842f5-c312-4e68-a679-c5a5e604d6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:29.436468 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1174/2367] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:29.441040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce842f5-c312-4e68-a679-c5a5e604d6d6 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.449986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce842f5-c312-4e68-a679-c5a5e604d6d6 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.456649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e4bea-2560-4fe7-9194-387a68d4acd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.464784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e4bea-2560-4fe7-9194-387a68d4acd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.464784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e4bea-2560-4fe7-9194-387a68d4acd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.464784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e4bea-2560-4fe7-9194-387a68d4acd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.464784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e4bea-2560-4fe7-9194-387a68d4acd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:59:29.464784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e4bea-2560-4fe7-9194-387a68d4acd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.464784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e4bea-2560-4fe7-9194-387a68d4acd9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.472368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1175/2368] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:59:29.476507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e4bea-2560-4fe7-9194-387a68d4acd9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.486328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e4bea-2560-4fe7-9194-387a68d4acd9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.491027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31f138-a79c-421c-a9e4-6025209a041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.491606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31f138-a79c-421c-a9e4-6025209a041c admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.491890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31f138-a79c-421c-a9e4-6025209a041c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.492168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31f138-a79c-421c-a9e4-6025209a041c admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.492784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31f138-a79c-421c-a9e4-6025209a041c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.494719 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31f138-a79c-421c-a9e4-6025209a041c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.563868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1194/2369] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:29 2022] POST /identity/v3/users => generated 308 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:29.565520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0913e1c-7712-4b83-ae5c-f778cb43092d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.567255 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a0913e1c-7712-4b83-ae5c-f778cb43092d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:29.579618 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0913e1c-7712-4b83-ae5c-f778cb43092d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.585683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746290aa-8dfc-437b-990f-19be71448bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.586150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746290aa-8dfc-437b-990f-19be71448bab admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.586439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746290aa-8dfc-437b-990f-19be71448bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.586766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746290aa-8dfc-437b-990f-19be71448bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.587434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746290aa-8dfc-437b-990f-19be71448bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.593267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1176/2370] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:29 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:29.597810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31f138-a79c-421c-a9e4-6025209a041c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.598342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746290aa-8dfc-437b-990f-19be71448bab admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.611274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1195/2371] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:29.615514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746290aa-8dfc-437b-990f-19be71448bab admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.625566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746290aa-8dfc-437b-990f-19be71448bab admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.665758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31f138-a79c-421c-a9e4-6025209a041c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.670608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a48c82-9d8b-4d0a-a3db-fbdc191a354c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.671314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a48c82-9d8b-4d0a-a3db-fbdc191a354c admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.671591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a48c82-9d8b-4d0a-a3db-fbdc191a354c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.671862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a48c82-9d8b-4d0a-a3db-fbdc191a354c admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.672654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a48c82-9d8b-4d0a-a3db-fbdc191a354c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.676227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a48c82-9d8b-4d0a-a3db-fbdc191a354c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.684755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f2050-4967-4170-8550-35cac05cbcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.687187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2050-4967-4170-8550-35cac05cbcda admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.687187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2050-4967-4170-8550-35cac05cbcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.687187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2050-4967-4170-8550-35cac05cbcda admin admin] PATH_INFO: `/v3/projects/c0f6f07ba2e04087a5451fe53889d621/users/79c5e39d10a44296b699c1f6b89e0e24/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.702575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f2050-4967-4170-8550-35cac05cbcda admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0f6f07ba2e04087a5451fe53889d621, user_id=79c5e39d10a44296b699c1f6b89e0e24, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.705936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f2050-4967-4170-8550-35cac05cbcda admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.822199 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1177/2372] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:29 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:29.826084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a48c82-9d8b-4d0a-a3db-fbdc191a354c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.836723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a48c82-9d8b-4d0a-a3db-fbdc191a354c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.841405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1cc88-be76-4b29-928e-b5420278f891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.841989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1cc88-be76-4b29-928e-b5420278f891 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.842267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1cc88-be76-4b29-928e-b5420278f891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.842565 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1cc88-be76-4b29-928e-b5420278f891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.843324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1cc88-be76-4b29-928e-b5420278f891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.846164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1cc88-be76-4b29-928e-b5420278f891 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.849843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:29.852046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1196/2373] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:29 2022] PUT /identity/v3/projects/c0f6f07ba2e04087a5451fe53889d621/users/79c5e39d10a44296b699c1f6b89e0e24/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:29.855960 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f2050-4967-4170-8550-35cac05cbcda admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.858612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1178/2374] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:29.862891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1cc88-be76-4b29-928e-b5420278f891 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.872893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1cc88-be76-4b29-928e-b5420278f891 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.882214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:29.920268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f2050-4967-4170-8550-35cac05cbcda admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.929393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919fcb0-f92f-48d1-a14a-4e6f6f20ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.930715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919fcb0-f92f-48d1-a14a-4e6f6f20ea59 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.931001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919fcb0-f92f-48d1-a14a-4e6f6f20ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.931280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919fcb0-f92f-48d1-a14a-4e6f6f20ea59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.932509 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919fcb0-f92f-48d1-a14a-4e6f6f20ea59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.934265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919fcb0-f92f-48d1-a14a-4e6f6f20ea59 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.949633 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1197/2375] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:29.949829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12514877-05fc-43a3-b790-09c95fb61b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.949829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12514877-05fc-43a3-b790-09c95fb61b26 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.949829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12514877-05fc-43a3-b790-09c95fb61b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.949829 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12514877-05fc-43a3-b790-09c95fb61b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.950584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12514877-05fc-43a3-b790-09c95fb61b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.950695 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919fcb0-f92f-48d1-a14a-4e6f6f20ea59 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.952238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12514877-05fc-43a3-b790-09c95fb61b26 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.961163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919fcb0-f92f-48d1-a14a-4e6f6f20ea59 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.965533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1179/2376] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:29.966264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4e41e-4b17-4a6d-b00c-bfecadc6d073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:29.968089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4e41e-4b17-4a6d-b00c-bfecadc6d073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:29.968089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4e41e-4b17-4a6d-b00c-bfecadc6d073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:29.968089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4e41e-4b17-4a6d-b00c-bfecadc6d073 admin admin] PATH_INFO: `/v3/projects/c0f6f07ba2e04087a5451fe53889d621/users/79c5e39d10a44296b699c1f6b89e0e24/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:29.969546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4e41e-4b17-4a6d-b00c-bfecadc6d073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0f6f07ba2e04087a5451fe53889d621, user_id=79c5e39d10a44296b699c1f6b89e0e24, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:29.971090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12514877-05fc-43a3-b790-09c95fb61b26 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:29.972650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4e41e-4b17-4a6d-b00c-bfecadc6d073 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:29.981077 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12514877-05fc-43a3-b790-09c95fb61b26 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:29.984094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1198/2377] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:29 2022] PUT /identity/v3/projects/c0f6f07ba2e04087a5451fe53889d621/users/79c5e39d10a44296b699c1f6b89e0e24/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:30.042830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e29a74-c907-4c64-9ebb-e8b78d6c07c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:30.043537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e29a74-c907-4c64-9ebb-e8b78d6c07c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:30.043821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e29a74-c907-4c64-9ebb-e8b78d6c07c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:30.044103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e29a74-c907-4c64-9ebb-e8b78d6c07c5 admin admin] PATH_INFO: `/v3/projects/d2fd21febf86475ca771363922e6f047/users/b1595e8c9ce842ef90c16b7f5351ee00/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:30.059188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e29a74-c907-4c64-9ebb-e8b78d6c07c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2fd21febf86475ca771363922e6f047, user_id=b1595e8c9ce842ef90c16b7f5351ee00, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:30.061917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e29a74-c907-4c64-9ebb-e8b78d6c07c5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:30.170130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1180/2378] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:29 2022] PUT /identity/v3/projects/d2fd21febf86475ca771363922e6f047/users/b1595e8c9ce842ef90c16b7f5351ee00/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:30.180475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0375cb-1c46-49c1-8175-bfef1d09c9a2 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:30.180789 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0375cb-1c46-49c1-8175-bfef1d09c9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:30.181071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0375cb-1c46-49c1-8175-bfef1d09c9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:30.209663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-cb0375cb-1c46-49c1-8175-bfef1d09c9a2 None None] MFA Rules not processed for user `b1595e8c9ce842ef90c16b7f5351ee00`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:30.210936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-cb0375cb-1c46-49c1-8175-bfef1d09c9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:30.277658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1199/2379] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:30.282620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e29a74-c907-4c64-9ebb-e8b78d6c07c5 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:30.350827 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e29a74-c907-4c64-9ebb-e8b78d6c07c5 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:30.355677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0903ff-89d9-4e34-8c3d-a0168b7ff251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:30.356286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0903ff-89d9-4e34-8c3d-a0168b7ff251 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:30.356668 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0903ff-89d9-4e34-8c3d-a0168b7ff251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:30.356835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0903ff-89d9-4e34-8c3d-a0168b7ff251 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:30.357641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0903ff-89d9-4e34-8c3d-a0168b7ff251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:30.359576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0903ff-89d9-4e34-8c3d-a0168b7ff251 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:30.463532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1181/2380] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:30 2022] POST /identity/v3/projects => generated 371 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:30.467674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0375cb-1c46-49c1-8175-bfef1d09c9a2 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:30.478895 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0375cb-1c46-49c1-8175-bfef1d09c9a2 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:30.543040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b273b1-9005-4da9-8696-acaab784394d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:30.543657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b273b1-9005-4da9-8696-acaab784394d admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:30.543939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b273b1-9005-4da9-8696-acaab784394d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:30.544211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b273b1-9005-4da9-8696-acaab784394d admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:30.544848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b273b1-9005-4da9-8696-acaab784394d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:30.546902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b273b1-9005-4da9-8696-acaab784394d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:30.651701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1200/2381] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:30 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:30.655679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0903ff-89d9-4e34-8c3d-a0168b7ff251 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:30.661920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:30.667638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0903ff-89d9-4e34-8c3d-a0168b7ff251 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:30.673191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aaa84c-76ba-4bd0-adaa-12474191478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:30.673913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaa84c-76ba-4bd0-adaa-12474191478f admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:30.674198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaa84c-76ba-4bd0-adaa-12474191478f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:30.674469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaa84c-76ba-4bd0-adaa-12474191478f admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:30.675210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aaa84c-76ba-4bd0-adaa-12474191478f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:30.677015 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aaa84c-76ba-4bd0-adaa-12474191478f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:30.692920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1182/2382] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:30.694261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b273b1-9005-4da9-8696-acaab784394d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:30.705630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b273b1-9005-4da9-8696-acaab784394d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:30.710553 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b32ab-2980-4dce-9dee-79ae6c463cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:30.711247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b32ab-2980-4dce-9dee-79ae6c463cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:30.711562 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b32ab-2980-4dce-9dee-79ae6c463cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:30.711816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b32ab-2980-4dce-9dee-79ae6c463cca admin admin] PATH_INFO: `/v3/projects/c468fe4703bf4d17a0d30f5bc047550b/users/5739676328fe4e7aac595aa168720018/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:30.728527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b32ab-2980-4dce-9dee-79ae6c463cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c468fe4703bf4d17a0d30f5bc047550b, user_id=5739676328fe4e7aac595aa168720018, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:30.731301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b32ab-2980-4dce-9dee-79ae6c463cca admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:30.789113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:30.794495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1201/2383] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:30 2022] PUT /identity/v3/projects/c468fe4703bf4d17a0d30f5bc047550b/users/5739676328fe4e7aac595aa168720018/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:30.798825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aaa84c-76ba-4bd0-adaa-12474191478f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:30.809773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aaa84c-76ba-4bd0-adaa-12474191478f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:30.872881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5ae89-ea69-4ba8-9102-8149f0e35820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:30.873492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5ae89-ea69-4ba8-9102-8149f0e35820 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:30.873772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5ae89-ea69-4ba8-9102-8149f0e35820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:30.874043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5ae89-ea69-4ba8-9102-8149f0e35820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:30.874725 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5ae89-ea69-4ba8-9102-8149f0e35820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:30.877322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5ae89-ea69-4ba8-9102-8149f0e35820 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:30.890851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1183/2384] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:30.892976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b32ab-2980-4dce-9dee-79ae6c463cca admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:30.903980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b32ab-2980-4dce-9dee-79ae6c463cca admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:30.909696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66cf59-62ce-489e-82b0-3891016ffea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:30.910492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66cf59-62ce-489e-82b0-3891016ffea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:30.910872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66cf59-62ce-489e-82b0-3891016ffea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:30.911218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66cf59-62ce-489e-82b0-3891016ffea8 admin admin] PATH_INFO: `/v3/projects/c468fe4703bf4d17a0d30f5bc047550b/users/5739676328fe4e7aac595aa168720018/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:30.913248 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66cf59-62ce-489e-82b0-3891016ffea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c468fe4703bf4d17a0d30f5bc047550b, user_id=5739676328fe4e7aac595aa168720018, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:30.916063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66cf59-62ce-489e-82b0-3891016ffea8 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:30.928648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1202/2385] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:30 2022] PUT /identity/v3/projects/c468fe4703bf4d17a0d30f5bc047550b/users/5739676328fe4e7aac595aa168720018/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:30.940370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b77de-8366-4b31-b012-7a12739230e4 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:30.940768 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b77de-8366-4b31-b012-7a12739230e4 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:30.941053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b77de-8366-4b31-b012-7a12739230e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:30.970186 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-734b77de-8366-4b31-b012-7a12739230e4 None None] MFA Rules not processed for user `5739676328fe4e7aac595aa168720018`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:30.971936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-734b77de-8366-4b31-b012-7a12739230e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:31.045922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1184/2386] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:31.057416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66cf59-62ce-489e-82b0-3891016ffea8 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:31.059781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7b66cf59-62ce-489e-82b0-3891016ffea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:31.121487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66cf59-62ce-489e-82b0-3891016ffea8 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:31.125401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98523026-b291-41a5-9c58-bb5f53922658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:31.126250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523026-b291-41a5-9c58-bb5f53922658 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:31.126619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523026-b291-41a5-9c58-bb5f53922658 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:31.126901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523026-b291-41a5-9c58-bb5f53922658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:31.136547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98523026-b291-41a5-9c58-bb5f53922658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:31.137981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98523026-b291-41a5-9c58-bb5f53922658 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:31.148804 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1203/2387] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:31.516980 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106678f-83d7-45e8-9f9f-f318a73c546f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:31.518358 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106678f-83d7-45e8-9f9f-f318a73c546f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:31.518901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106678f-83d7-45e8-9f9f-f318a73c546f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:31.565836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7106678f-83d7-45e8-9f9f-f318a73c546f None None] MFA Rules not processed for user `79c5e39d10a44296b699c1f6b89e0e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:31.567936 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7106678f-83d7-45e8-9f9f-f318a73c546f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:31.636167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:31.644452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1185/2388] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:31 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:32.177573 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98523026-b291-41a5-9c58-bb5f53922658 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:32.247252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98523026-b291-41a5-9c58-bb5f53922658 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:32.252957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:32.254604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:32.254604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:32.254604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] PATH_INFO: `/v3/users/b1595e8c9ce842ef90c16b7f5351ee00` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:32.255596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1595e8c9ce842ef90c16b7f5351ee00)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:32.257528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:32.441350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1595e8c9ce842ef90c16b7f5351ee00'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:32.441674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1595e8c9ce842ef90c16b7f5351ee00'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:32.513400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1595e8c9ce842ef90c16b7f5351ee00'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:32.521533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1595e8c9ce842ef90c16b7f5351ee00'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:32.534067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1595e8c9ce842ef90c16b7f5351ee00'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:32.540149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1204/2389] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:32 2022] DELETE /identity/v3/users/b1595e8c9ce842ef90c16b7f5351ee00 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:32.544761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24a347-0228-47ef-83b0-7f46ced60e6c None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:32.545248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24a347-0228-47ef-83b0-7f46ced60e6c None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:32.545345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24a347-0228-47ef-83b0-7f46ced60e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:32.566618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2b24a347-0228-47ef-83b0-7f46ced60e6c None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:32.567859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2b24a347-0228-47ef-83b0-7f46ced60e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:32.636321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1186/2390] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:32.644958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:32.646538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:32.662040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:32.712213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4d156-2275-4091-9afc-cbc0b3709d37 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:32.715835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8d6ec-f6ac-4322-aeb1-1b86f4327405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:32.716471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8d6ec-f6ac-4322-aeb1-1b86f4327405 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:32.716859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8d6ec-f6ac-4322-aeb1-1b86f4327405 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:32.717142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8d6ec-f6ac-4322-aeb1-1b86f4327405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:32.727704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8d6ec-f6ac-4322-aeb1-1b86f4327405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:32.729335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8d6ec-f6ac-4322-aeb1-1b86f4327405 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:32.739485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1205/2391] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:59:33.083722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24a347-0228-47ef-83b0-7f46ced60e6c None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:33.092913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2b24a347-0228-47ef-83b0-7f46ced60e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:33.107923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24a347-0228-47ef-83b0-7f46ced60e6c None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:33.112564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:33.113233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:33.113511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:33.113788 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] PATH_INFO: `/v3/users/bdee2ddc3b70463297d617faeb8f776c` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:33.114946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdee2ddc3b70463297d617faeb8f776c)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:33.116887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:33.200438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8d6ec-f6ac-4322-aeb1-1b86f4327405 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:33.202527 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-52b8d6ec-f6ac-4322-aeb1-1b86f4327405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:33.218800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8d6ec-f6ac-4322-aeb1-1b86f4327405 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:33.224064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:33.224708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:33.224991 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:33.225337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] PATH_INFO: `/v3/projects/d2fd21febf86475ca771363922e6f047` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:33.226539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2fd21febf86475ca771363922e6f047)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:33.228345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:33.317014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdee2ddc3b70463297d617faeb8f776c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.321324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2fd21febf86475ca771363922e6f047', 'request_id': 'req-922aee65-02e7-4ddb-bee0-eaf54d47ad04'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.327183 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2fd21febf86475ca771363922e6f047', 'request_id': 'req-922aee65-02e7-4ddb-bee0-eaf54d47ad04'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.376699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdee2ddc3b70463297d617faeb8f776c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.389508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdee2ddc3b70463297d617faeb8f776c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.396457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdee2ddc3b70463297d617faeb8f776c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.397110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdee2ddc3b70463297d617faeb8f776c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.403422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1187/2392] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:33 2022] DELETE /identity/v3/users/bdee2ddc3b70463297d617faeb8f776c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:33.440087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2fd21febf86475ca771363922e6f047', 'request_id': 'req-922aee65-02e7-4ddb-bee0-eaf54d47ad04'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.454555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] The token cache is being invalidate because project d2fd21febf86475ca771363922e6f047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:33.454870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.455540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.457154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1206/2393] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:33 2022] DELETE /identity/v3/projects/d2fd21febf86475ca771363922e6f047 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:33.463626 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:33.465525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:33.545636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3562c-ac32-4e02-b182-a387621f82cc admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:33.552552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:33.553286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:33.553636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:33.554023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] PATH_INFO: `/v3/users/5739676328fe4e7aac595aa168720018` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:33.555391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5739676328fe4e7aac595aa168720018)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:33.557474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:33.572471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:33.726723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:33.728465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:33.804791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922aee65-02e7-4ddb-bee0-eaf54d47ad04 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:33.809422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:33.810057 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:33.810345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:33.810649 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] PATH_INFO: `/v3/projects/480e1e85ebb84605920bff6e077ea7ce` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:33.811779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=480e1e85ebb84605920bff6e077ea7ce)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:33.814220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:33.863155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5739676328fe4e7aac595aa168720018'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.933710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:33.943195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480e1e85ebb84605920bff6e077ea7ce', 'request_id': 'req-3963bcbd-be03-46f0-a91d-0f088892cff3'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.948572 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480e1e85ebb84605920bff6e077ea7ce', 'request_id': 'req-3963bcbd-be03-46f0-a91d-0f088892cff3'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.952787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5739676328fe4e7aac595aa168720018'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.965998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5739676328fe4e7aac595aa168720018'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.974192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5739676328fe4e7aac595aa168720018'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.974904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5739676328fe4e7aac595aa168720018'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:33.980665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1188/2394] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:33 2022] DELETE /identity/v3/users/5739676328fe4e7aac595aa168720018 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:34.050846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480e1e85ebb84605920bff6e077ea7ce', 'request_id': 'req-3963bcbd-be03-46f0-a91d-0f088892cff3'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.068362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] The token cache is being invalidate because project 480e1e85ebb84605920bff6e077ea7ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:34.068837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.069703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.071644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1207/2395] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:33 2022] DELETE /identity/v3/projects/480e1e85ebb84605920bff6e077ea7ce => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:34.075896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:34.077300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:34.143163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dfd5a-4604-4fcc-9533-4546b9fc9491 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:34.148143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:34.148811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:34.149096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:34.149374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] PATH_INFO: `/v3/users/79c5e39d10a44296b699c1f6b89e0e24` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:34.150520 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c5e39d10a44296b699c1f6b89e0e24)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:34.152662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:34.385750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c5e39d10a44296b699c1f6b89e0e24'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.452541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:34.455191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:34.484836 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c5e39d10a44296b699c1f6b89e0e24'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.499243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c5e39d10a44296b699c1f6b89e0e24'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.508160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c5e39d10a44296b699c1f6b89e0e24'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.508906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c5e39d10a44296b699c1f6b89e0e24'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.515357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1189/2396] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:34 2022] DELETE /identity/v3/users/79c5e39d10a44296b699c1f6b89e0e24 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:34.526765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963bcbd-be03-46f0-a91d-0f088892cff3 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:34.590144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:34.590824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:34.591156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:34.591556 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] PATH_INFO: `/v3/projects/c468fe4703bf4d17a0d30f5bc047550b` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:34.592795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] RBAC: Authorizing `identity:delete_project(project_id=c468fe4703bf4d17a0d30f5bc047550b)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:34.595613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:34.708602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c468fe4703bf4d17a0d30f5bc047550b', 'request_id': 'req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.714256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c468fe4703bf4d17a0d30f5bc047550b', 'request_id': 'req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.794227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:34.796130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:34.814133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4361d58-7fad-4033-b16e-779d7cdcc5d7 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:34.815298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c468fe4703bf4d17a0d30f5bc047550b', 'request_id': 'req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.821198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:34.831370 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] The token cache is being invalidate because project c468fe4703bf4d17a0d30f5bc047550b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:34.831693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.832361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8dfc56-dc5a-47f8-bbc5-d226e903cead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:34.833876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1208/2397] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:34 2022] DELETE /identity/v3/projects/c468fe4703bf4d17a0d30f5bc047550b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:34.838682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9112d4-a1ef-46aa-aefb-af5425c7cd83 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:34.838969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9112d4-a1ef-46aa-aefb-af5425c7cd83 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:34.839253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9112d4-a1ef-46aa-aefb-af5425c7cd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:34.848357 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:34.861477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-2a9112d4-a1ef-46aa-aefb-af5425c7cd83 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:34.862745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2a9112d4-a1ef-46aa-aefb-af5425c7cd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:34.879443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:34.880363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:34.880671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:34.880940 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] PATH_INFO: `/v3/projects/c0f6f07ba2e04087a5451fe53889d621` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:34.882225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0f6f07ba2e04087a5451fe53889d621)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:34.884678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:34.926351 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1209/2398] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:34.934129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b7ebe-200d-4e06-8dcd-c3776fc13bab None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:34.934485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b7ebe-200d-4e06-8dcd-c3776fc13bab None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:34.934817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b7ebe-200d-4e06-8dcd-c3776fc13bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:34.956681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-870b7ebe-200d-4e06-8dcd-c3776fc13bab None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:34.957947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-870b7ebe-200d-4e06-8dcd-c3776fc13bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:34.966262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1210/2399] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:34.971448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b7ebe-200d-4e06-8dcd-c3776fc13bab None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:34.980520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b7ebe-200d-4e06-8dcd-c3776fc13bab None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:34.985182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222553-8b08-4801-84f4-48dd9b00041b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:34.985850 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222553-8b08-4801-84f4-48dd9b00041b admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:34.986184 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222553-8b08-4801-84f4-48dd9b00041b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:34.986467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222553-8b08-4801-84f4-48dd9b00041b admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:34.987148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc222553-8b08-4801-84f4-48dd9b00041b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:59:34.987445 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc222553-8b08-4801-84f4-48dd9b00041b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:34.989039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc222553-8b08-4801-84f4-48dd9b00041b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.003076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1211/2400] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:59:35.009275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222553-8b08-4801-84f4-48dd9b00041b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.013201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0f6f07ba2e04087a5451fe53889d621', 'request_id': 'req-831cec6d-b2be-422d-9dc6-bf78f6aedb24'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:35.021922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222553-8b08-4801-84f4-48dd9b00041b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.077059 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271c1f2-e31b-4c3c-9509-b7417040ff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.077601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271c1f2-e31b-4c3c-9509-b7417040ff51 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.077891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271c1f2-e31b-4c3c-9509-b7417040ff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.078165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271c1f2-e31b-4c3c-9509-b7417040ff51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.079420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271c1f2-e31b-4c3c-9509-b7417040ff51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.081231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271c1f2-e31b-4c3c-9509-b7417040ff51 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.096823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0f6f07ba2e04087a5451fe53889d621', 'request_id': 'req-831cec6d-b2be-422d-9dc6-bf78f6aedb24'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:35.103545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0f6f07ba2e04087a5451fe53889d621', 'request_id': 'req-831cec6d-b2be-422d-9dc6-bf78f6aedb24'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:35.114254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] The token cache is being invalidate because project c0f6f07ba2e04087a5451fe53889d621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:35.114627 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:35.115264 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-831cec6d-b2be-422d-9dc6-bf78f6aedb24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:35.116766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1190/2401] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:34 2022] DELETE /identity/v3/projects/c0f6f07ba2e04087a5451fe53889d621 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:35.122304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327534ac-d26e-4e79-9a40-fb095c788c92 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.122528 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327534ac-d26e-4e79-9a40-fb095c788c92 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.122812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327534ac-d26e-4e79-9a40-fb095c788c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.142880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-327534ac-d26e-4e79-9a40-fb095c788c92 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:35.144077 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-327534ac-d26e-4e79-9a40-fb095c788c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:35.151895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1191/2402] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:35.160124 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3896b8-d8bc-433b-a71e-9a3c20223fbe None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.160440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3896b8-d8bc-433b-a71e-9a3c20223fbe None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.160648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3896b8-d8bc-433b-a71e-9a3c20223fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.176686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1212/2403] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:35.181574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-eb3896b8-d8bc-433b-a71e-9a3c20223fbe None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:35.181715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271c1f2-e31b-4c3c-9509-b7417040ff51 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.183560 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-eb3896b8-d8bc-433b-a71e-9a3c20223fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:35.183951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e271c1f2-e31b-4c3c-9509-b7417040ff51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:35.251136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1192/2404] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:35.255622 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3896b8-d8bc-433b-a71e-9a3c20223fbe None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.258405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271c1f2-e31b-4c3c-9509-b7417040ff51 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.263261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79369b02-27d4-4960-a23a-5c73d5cf8f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.263892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79369b02-27d4-4960-a23a-5c73d5cf8f20 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.264178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79369b02-27d4-4960-a23a-5c73d5cf8f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.264455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79369b02-27d4-4960-a23a-5c73d5cf8f20 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.264573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3896b8-d8bc-433b-a71e-9a3c20223fbe None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.265090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79369b02-27d4-4960-a23a-5c73d5cf8f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.267841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79369b02-27d4-4960-a23a-5c73d5cf8f20 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.270026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe25e1-9a95-446a-9cc4-57bbfff3949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.270656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe25e1-9a95-446a-9cc4-57bbfff3949c admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.270944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe25e1-9a95-446a-9cc4-57bbfff3949c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.271223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe25e1-9a95-446a-9cc4-57bbfff3949c admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.271880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe25e1-9a95-446a-9cc4-57bbfff3949c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:59:35.272192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe25e1-9a95-446a-9cc4-57bbfff3949c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.273807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe25e1-9a95-446a-9cc4-57bbfff3949c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.287029 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1193/2405] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:59:35.291171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe25e1-9a95-446a-9cc4-57bbfff3949c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.301747 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe25e1-9a95-446a-9cc4-57bbfff3949c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.306426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0e0f8-a335-4a90-a848-d58caee0ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.307028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0e0f8-a335-4a90-a848-d58caee0ce59 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.307314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0e0f8-a335-4a90-a848-d58caee0ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.307587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0e0f8-a335-4a90-a848-d58caee0ce59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.308194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0e0f8-a335-4a90-a848-d58caee0ce59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.310079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0e0f8-a335-4a90-a848-d58caee0ce59 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.342130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:35.348768 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1213/2406] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/users => generated 281 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:35.352864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79369b02-27d4-4960-a23a-5c73d5cf8f20 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.362969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79369b02-27d4-4960-a23a-5c73d5cf8f20 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.367551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164ffab-6ae6-4547-a85f-6d51facf8e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.368207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164ffab-6ae6-4547-a85f-6d51facf8e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.368505 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164ffab-6ae6-4547-a85f-6d51facf8e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.368779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164ffab-6ae6-4547-a85f-6d51facf8e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.369418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164ffab-6ae6-4547-a85f-6d51facf8e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.371049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164ffab-6ae6-4547-a85f-6d51facf8e66 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.381795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1214/2407] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:35.385528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164ffab-6ae6-4547-a85f-6d51facf8e66 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.395810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164ffab-6ae6-4547-a85f-6d51facf8e66 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.400451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90b452-aef1-4f5b-8bb5-0360ef71c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.401018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90b452-aef1-4f5b-8bb5-0360ef71c42d admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.401298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90b452-aef1-4f5b-8bb5-0360ef71c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.402276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90b452-aef1-4f5b-8bb5-0360ef71c42d admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.402276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90b452-aef1-4f5b-8bb5-0360ef71c42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.403754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90b452-aef1-4f5b-8bb5-0360ef71c42d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.414084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1215/2408] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:35.418033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90b452-aef1-4f5b-8bb5-0360ef71c42d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.422118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:35.427704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90b452-aef1-4f5b-8bb5-0360ef71c42d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.429721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1194/2409] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:35.433778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0e0f8-a335-4a90-a848-d58caee0ce59 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.487941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354c9ac-8a6e-4063-8ddb-79562a0cca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.488680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354c9ac-8a6e-4063-8ddb-79562a0cca92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.488935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354c9ac-8a6e-4063-8ddb-79562a0cca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.489213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354c9ac-8a6e-4063-8ddb-79562a0cca92 admin admin] PATH_INFO: `/v3/projects/9918ac1e9007498abbdfccce46edf4cf/users/b4dd4d33280d4284a4e0bd673ac72641/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.496770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0e0f8-a335-4a90-a848-d58caee0ce59 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.502477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ae0c0-1e7f-40c7-9cb1-000653ad327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.504204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ae0c0-1e7f-40c7-9cb1-000653ad327e admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.504204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ae0c0-1e7f-40c7-9cb1-000653ad327e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.504204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ae0c0-1e7f-40c7-9cb1-000653ad327e admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.504604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ae0c0-1e7f-40c7-9cb1-000653ad327e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.505514 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354c9ac-8a6e-4063-8ddb-79562a0cca92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9918ac1e9007498abbdfccce46edf4cf, user_id=b4dd4d33280d4284a4e0bd673ac72641, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.506576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ae0c0-1e7f-40c7-9cb1-000653ad327e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.508224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354c9ac-8a6e-4063-8ddb-79562a0cca92 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.634655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1216/2410] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:35 2022] PUT /identity/v3/projects/9918ac1e9007498abbdfccce46edf4cf/users/b4dd4d33280d4284a4e0bd673ac72641/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:35.644883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179ebe1-c949-4e56-9943-9c580eae9ee4 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.645176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179ebe1-c949-4e56-9943-9c580eae9ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.645460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179ebe1-c949-4e56-9943-9c580eae9ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.673593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d179ebe1-c949-4e56-9943-9c580eae9ee4 None None] MFA Rules not processed for user `b4dd4d33280d4284a4e0bd673ac72641`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:35.674950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d179ebe1-c949-4e56-9943-9c580eae9ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:35.681253 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1195/2411] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/users => generated 294 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:35.692201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ae0c0-1e7f-40c7-9cb1-000653ad327e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.702055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ae0c0-1e7f-40c7-9cb1-000653ad327e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.743378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1217/2412] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:35.747898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179ebe1-c949-4e56-9943-9c580eae9ee4 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.762174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a0359-07a5-4014-8dba-5170fede6c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.762859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a0359-07a5-4014-8dba-5170fede6c30 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.763141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a0359-07a5-4014-8dba-5170fede6c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.763418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a0359-07a5-4014-8dba-5170fede6c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.764104 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a0359-07a5-4014-8dba-5170fede6c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.766198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a0359-07a5-4014-8dba-5170fede6c30 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.780209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1196/2413] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:35.781232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a0359-07a5-4014-8dba-5170fede6c30 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.791688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a0359-07a5-4014-8dba-5170fede6c30 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.796809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4e6e8-f529-4474-b03d-c6c2dc4efe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.798006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:35.798122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4e6e8-f529-4474-b03d-c6c2dc4efe8f admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.798483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4e6e8-f529-4474-b03d-c6c2dc4efe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.798905 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4e6e8-f529-4474-b03d-c6c2dc4efe8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.799585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4e6e8-f529-4474-b03d-c6c2dc4efe8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.801233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4e6e8-f529-4474-b03d-c6c2dc4efe8f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.816000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1197/2414] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:35.816180 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179ebe1-c949-4e56-9943-9c580eae9ee4 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.817053 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4e6e8-f529-4474-b03d-c6c2dc4efe8f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.820306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfd156-7e8d-4832-87e4-e1160640718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.820923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfd156-7e8d-4832-87e4-e1160640718b admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.821207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfd156-7e8d-4832-87e4-e1160640718b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.821478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfd156-7e8d-4832-87e4-e1160640718b admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.822110 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfd156-7e8d-4832-87e4-e1160640718b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.824134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfd156-7e8d-4832-87e4-e1160640718b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.827105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4e6e8-f529-4474-b03d-c6c2dc4efe8f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.832371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd7bef-db1b-456c-a4c5-fecdd03dfd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:35.833393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd7bef-db1b-456c-a4c5-fecdd03dfd36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.833393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd7bef-db1b-456c-a4c5-fecdd03dfd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.833632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd7bef-db1b-456c-a4c5-fecdd03dfd36 admin admin] PATH_INFO: `/v3/projects/9f61c1a0c4d24b53a23e271a09b058eb/users/899ae20deec54534ab419cf552066f29/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.848434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd7bef-db1b-456c-a4c5-fecdd03dfd36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f61c1a0c4d24b53a23e271a09b058eb, user_id=899ae20deec54534ab419cf552066f29, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:35.851315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd7bef-db1b-456c-a4c5-fecdd03dfd36 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:35.892279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1218/2415] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:35.896346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfd156-7e8d-4832-87e4-e1160640718b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:35.914147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1198/2416] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:35 2022] PUT /identity/v3/projects/9f61c1a0c4d24b53a23e271a09b058eb/users/899ae20deec54534ab419cf552066f29/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:35.926996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6707e3-b470-424a-a026-b01b4cc44ff8 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:35.927283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6707e3-b470-424a-a026-b01b4cc44ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:35.927647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6707e3-b470-424a-a026-b01b4cc44ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:35.956867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-3a6707e3-b470-424a-a026-b01b4cc44ff8 None None] MFA Rules not processed for user `899ae20deec54534ab419cf552066f29`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:35.958108 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3a6707e3-b470-424a-a026-b01b4cc44ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:35.964219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfd156-7e8d-4832-87e4-e1160640718b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:35.997974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:36.033942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1199/2417] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:36.038243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6707e3-b470-424a-a026-b01b4cc44ff8 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.042641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720db2e-204d-47c2-9a6a-e3da006e2bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.043394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720db2e-204d-47c2-9a6a-e3da006e2bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.043550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720db2e-204d-47c2-9a6a-e3da006e2bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.043823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720db2e-204d-47c2-9a6a-e3da006e2bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.044899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720db2e-204d-47c2-9a6a-e3da006e2bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.048774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720db2e-204d-47c2-9a6a-e3da006e2bb4 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.050140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6707e3-b470-424a-a026-b01b4cc44ff8 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.055304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7f490-7239-4df1-b67c-ca85b7e80055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.055937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7f490-7239-4df1-b67c-ca85b7e80055 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.056225 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7f490-7239-4df1-b67c-ca85b7e80055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.056504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7f490-7239-4df1-b67c-ca85b7e80055 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.057157 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7f490-7239-4df1-b67c-ca85b7e80055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.059280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7f490-7239-4df1-b67c-ca85b7e80055 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.170386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1219/2418] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:35 2022] POST /identity/v3/users => generated 289 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:36.175291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720db2e-204d-47c2-9a6a-e3da006e2bb4 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.184938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720db2e-204d-47c2-9a6a-e3da006e2bb4 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.190238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f8f82-cc73-44ca-8555-b512f4054ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.190863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f8f82-cc73-44ca-8555-b512f4054ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.191152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f8f82-cc73-44ca-8555-b512f4054ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.191431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f8f82-cc73-44ca-8555-b512f4054ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.192079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f8f82-cc73-44ca-8555-b512f4054ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.193701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f8f82-cc73-44ca-8555-b512f4054ddc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.204668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1220/2419] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:36.208300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1200/2420] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:36 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:36.215229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f8f82-cc73-44ca-8555-b512f4054ddc admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.215685 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7f490-7239-4df1-b67c-ca85b7e80055 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.225853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f8f82-cc73-44ca-8555-b512f4054ddc admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.294096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7f490-7239-4df1-b67c-ca85b7e80055 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.300014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a16cd-b00c-48b2-a887-80ec0409e397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.300660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a16cd-b00c-48b2-a887-80ec0409e397 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.301011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a16cd-b00c-48b2-a887-80ec0409e397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.301317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a16cd-b00c-48b2-a887-80ec0409e397 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.301980 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a16cd-b00c-48b2-a887-80ec0409e397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.304022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a16cd-b00c-48b2-a887-80ec0409e397 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.403866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1201/2421] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:36 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:36.404929 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a16cd-b00c-48b2-a887-80ec0409e397 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.416349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a16cd-b00c-48b2-a887-80ec0409e397 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.419690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b9088-1fdc-45f5-9814-6f910921be53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.420262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b9088-1fdc-45f5-9814-6f910921be53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.420602 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b9088-1fdc-45f5-9814-6f910921be53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.422242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b9088-1fdc-45f5-9814-6f910921be53 admin admin] PATH_INFO: `/v3/projects/e30bf7701f8e4b8c8277df525a9099e6/users/5bd42a3e0eb0429e95225411a9e59a51/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.423989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dafe2d-1c5d-4426-85f8-f643668b59a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.424141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:36.424604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dafe2d-1c5d-4426-85f8-f643668b59a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.424891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dafe2d-1c5d-4426-85f8-f643668b59a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.425156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dafe2d-1c5d-4426-85f8-f643668b59a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.425810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dafe2d-1c5d-4426-85f8-f643668b59a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.427458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dafe2d-1c5d-4426-85f8-f643668b59a1 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.438489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1202/2422] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:36.438825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b9088-1fdc-45f5-9814-6f910921be53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30bf7701f8e4b8c8277df525a9099e6, user_id=5bd42a3e0eb0429e95225411a9e59a51, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.441539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b9088-1fdc-45f5-9814-6f910921be53 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.445657 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dafe2d-1c5d-4426-85f8-f643668b59a1 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.460050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dafe2d-1c5d-4426-85f8-f643668b59a1 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.464982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288c669-a75e-47a5-b11e-b12d43f9bd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.465666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288c669-a75e-47a5-b11e-b12d43f9bd51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.465953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288c669-a75e-47a5-b11e-b12d43f9bd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.466235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288c669-a75e-47a5-b11e-b12d43f9bd51 admin admin] PATH_INFO: `/v3/projects/fbb890875f904cbdad4f7bf8afa6384e/users/8b4c516d03e24fe1872b5ae584dbb1aa/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.482647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288c669-a75e-47a5-b11e-b12d43f9bd51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb890875f904cbdad4f7bf8afa6384e, user_id=8b4c516d03e24fe1872b5ae584dbb1aa, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.485708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288c669-a75e-47a5-b11e-b12d43f9bd51 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.554784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1221/2423] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:36 2022] PUT /identity/v3/projects/e30bf7701f8e4b8c8277df525a9099e6/users/5bd42a3e0eb0429e95225411a9e59a51/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:36.557418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b9088-1fdc-45f5-9814-6f910921be53 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.584620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1203/2424] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:36 2022] PUT /identity/v3/projects/fbb890875f904cbdad4f7bf8afa6384e/users/8b4c516d03e24fe1872b5ae584dbb1aa/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:36.588911 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288c669-a75e-47a5-b11e-b12d43f9bd51 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.630857 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b9088-1fdc-45f5-9814-6f910921be53 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.676179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288c669-a75e-47a5-b11e-b12d43f9bd51 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.680993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525799a-e30e-4401-b8e0-d3a9fa3cd07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.681600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525799a-e30e-4401-b8e0-d3a9fa3cd07c admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.681883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525799a-e30e-4401-b8e0-d3a9fa3cd07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.682160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525799a-e30e-4401-b8e0-d3a9fa3cd07c admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.682846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525799a-e30e-4401-b8e0-d3a9fa3cd07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.684477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525799a-e30e-4401-b8e0-d3a9fa3cd07c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.692832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01b005-3057-4ef2-89a6-21303ea5d3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.693441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01b005-3057-4ef2-89a6-21303ea5d3df admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.693720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01b005-3057-4ef2-89a6-21303ea5d3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.693992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01b005-3057-4ef2-89a6-21303ea5d3df admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.694856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01b005-3057-4ef2-89a6-21303ea5d3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.698311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1204/2425] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:36.698453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01b005-3057-4ef2-89a6-21303ea5d3df admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.699354 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525799a-e30e-4401-b8e0-d3a9fa3cd07c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.707521 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1222/2426] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:36.711688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01b005-3057-4ef2-89a6-21303ea5d3df admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.711813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525799a-e30e-4401-b8e0-d3a9fa3cd07c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.716409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9ebdd-9a1d-4415-82fe-ef5a2a82fd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.717283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9ebdd-9a1d-4415-82fe-ef5a2a82fd25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.717576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9ebdd-9a1d-4415-82fe-ef5a2a82fd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.717853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9ebdd-9a1d-4415-82fe-ef5a2a82fd25 admin admin] PATH_INFO: `/v3/projects/fbb890875f904cbdad4f7bf8afa6384e/users/8b4c516d03e24fe1872b5ae584dbb1aa/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.719888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9ebdd-9a1d-4415-82fe-ef5a2a82fd25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb890875f904cbdad4f7bf8afa6384e, user_id=8b4c516d03e24fe1872b5ae584dbb1aa, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.721629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01b005-3057-4ef2-89a6-21303ea5d3df admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.722555 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9ebdd-9a1d-4415-82fe-ef5a2a82fd25 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.724882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:36.726601 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb6903-0aec-4932-8843-19477d2d664e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.730487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb6903-0aec-4932-8843-19477d2d664e admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.730487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb6903-0aec-4932-8843-19477d2d664e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.730487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb6903-0aec-4932-8843-19477d2d664e admin admin] PATH_INFO: `/v3/projects/e30bf7701f8e4b8c8277df525a9099e6/users/5bd42a3e0eb0429e95225411a9e59a51/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.730487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb6903-0aec-4932-8843-19477d2d664e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30bf7701f8e4b8c8277df525a9099e6, user_id=5bd42a3e0eb0429e95225411a9e59a51, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.732437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb6903-0aec-4932-8843-19477d2d664e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.733913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1205/2427] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:36 2022] PUT /identity/v3/projects/fbb890875f904cbdad4f7bf8afa6384e/users/8b4c516d03e24fe1872b5ae584dbb1aa/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:36.743801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1223/2428] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:36 2022] PUT /identity/v3/projects/e30bf7701f8e4b8c8277df525a9099e6/users/5bd42a3e0eb0429e95225411a9e59a51/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:36.744809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533faa58-1ac5-453c-9a82-e50454253d45 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.745148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533faa58-1ac5-453c-9a82-e50454253d45 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.745532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533faa58-1ac5-453c-9a82-e50454253d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.756135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edcc05-a8f0-4778-8694-456d6ed83f6b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.756421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edcc05-a8f0-4778-8694-456d6ed83f6b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.756694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edcc05-a8f0-4778-8694-456d6ed83f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.779850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-533faa58-1ac5-453c-9a82-e50454253d45 None None] MFA Rules not processed for user `8b4c516d03e24fe1872b5ae584dbb1aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:36.781062 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-533faa58-1ac5-453c-9a82-e50454253d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:36.784758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-92edcc05-a8f0-4778-8694-456d6ed83f6b None None] MFA Rules not processed for user `5bd42a3e0eb0429e95225411a9e59a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:36.786414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-92edcc05-a8f0-4778-8694-456d6ed83f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:36.860960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1206/2429] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:36.863466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:36.869237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1224/2430] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:36.874309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533faa58-1ac5-453c-9a82-e50454253d45 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.876375 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-533faa58-1ac5-453c-9a82-e50454253d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:36.883473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edcc05-a8f0-4778-8694-456d6ed83f6b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:36.884831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-92edcc05-a8f0-4778-8694-456d6ed83f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:36.935447 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533faa58-1ac5-453c-9a82-e50454253d45 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.940357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd235d2-22dc-4c55-b362-495079a95042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.940989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd235d2-22dc-4c55-b362-495079a95042 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.941257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd235d2-22dc-4c55-b362-495079a95042 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.941538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd235d2-22dc-4c55-b362-495079a95042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.947587 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edcc05-a8f0-4778-8694-456d6ed83f6b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:36.951687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6badb-24ea-450e-b640-9735f806f0ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:36.951826 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd235d2-22dc-4c55-b362-495079a95042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.952574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6badb-24ea-450e-b640-9735f806f0ff service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:36.952863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6badb-24ea-450e-b640-9735f806f0ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:36.953196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6badb-24ea-450e-b640-9735f806f0ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:36.953305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd235d2-22dc-4c55-b362-495079a95042 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.964357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1207/2431] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:36.964808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6badb-24ea-450e-b640-9735f806f0ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:36.968455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6badb-24ea-450e-b640-9735f806f0ff service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:36.977817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1225/2432] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:59:36.995722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd235d2-22dc-4c55-b362-495079a95042 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:37.068649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd235d2-22dc-4c55-b362-495079a95042 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:37.079471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:37.080153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:37.080440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:37.080718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] PATH_INFO: `/v3/users/899ae20deec54534ab419cf552066f29` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:37.082020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=899ae20deec54534ab419cf552066f29)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:37.084029 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:37.236155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6badb-24ea-450e-b640-9735f806f0ff service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:37.244613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6badb-24ea-450e-b640-9735f806f0ff service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:37.248168 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a5793-8646-4cf5-9f52-1d4320257f13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:37.248754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a5793-8646-4cf5-9f52-1d4320257f13 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:37.249028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a5793-8646-4cf5-9f52-1d4320257f13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:37.249299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a5793-8646-4cf5-9f52-1d4320257f13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:37.303620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '899ae20deec54534ab419cf552066f29'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:37.308647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:INSERT=1,keystone:SELECT=95,keystone:DELETE=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:37.311407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a5793-8646-4cf5-9f52-1d4320257f13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:37.313525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a5793-8646-4cf5-9f52-1d4320257f13 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:37.322701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1226/2433] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:59:37.376201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '899ae20deec54534ab419cf552066f29'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:37.389368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '899ae20deec54534ab419cf552066f29'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:37.396681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '899ae20deec54534ab419cf552066f29'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:37.397415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '899ae20deec54534ab419cf552066f29'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:37.403324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1208/2434] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:36 2022] DELETE /identity/v3/users/899ae20deec54534ab419cf552066f29 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:37.409537 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708ef4f-ca38-4e60-88a9-d8be43f21c1d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:37.409827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708ef4f-ca38-4e60-88a9-d8be43f21c1d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:37.410098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708ef4f-ca38-4e60-88a9-d8be43f21c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:37.428995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a708ef4f-ca38-4e60-88a9-d8be43f21c1d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:37.430191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a708ef4f-ca38-4e60-88a9-d8be43f21c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:37.491247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1227/2435] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:37.501601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:37.507501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:37.564007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf4b28-9403-4cda-885d-2c9379e85ca4 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:37.568248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6aa1-4f8d-4b26-8abc-18247dcf1e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:37.568868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6aa1-4f8d-4b26-8abc-18247dcf1e27 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:37.569151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6aa1-4f8d-4b26-8abc-18247dcf1e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:37.569427 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6aa1-4f8d-4b26-8abc-18247dcf1e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:37.579147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac6aa1-4f8d-4b26-8abc-18247dcf1e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:37.580882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac6aa1-4f8d-4b26-8abc-18247dcf1e27 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:37.590433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1209/2436] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:38.025716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708ef4f-ca38-4e60-88a9-d8be43f21c1d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:38.027437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a708ef4f-ca38-4e60-88a9-d8be43f21c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:38.053778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708ef4f-ca38-4e60-88a9-d8be43f21c1d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:38.060303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:38.060964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:38.061249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:38.061640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] PATH_INFO: `/v3/projects/9f61c1a0c4d24b53a23e271a09b058eb` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:38.062923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f61c1a0c4d24b53a23e271a09b058eb)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:38.064848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:38.173773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:38.184271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f61c1a0c4d24b53a23e271a09b058eb', 'request_id': 'req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.190858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f61c1a0c4d24b53a23e271a09b058eb', 'request_id': 'req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.272426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f61c1a0c4d24b53a23e271a09b058eb', 'request_id': 'req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.285630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] The token cache is being invalidate because project 9f61c1a0c4d24b53a23e271a09b058eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:38.285926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.286626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.288137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1228/2437] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:38 2022] DELETE /identity/v3/projects/9f61c1a0c4d24b53a23e271a09b058eb => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:38.292436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6aa1-4f8d-4b26-8abc-18247dcf1e27 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:38.303410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6aa1-4f8d-4b26-8abc-18247dcf1e27 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:38.367769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:38.368672 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:38.368954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:38.369227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] PATH_INFO: `/v3/users/8b4c516d03e24fe1872b5ae584dbb1aa` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:38.370393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4c516d03e24fe1872b5ae584dbb1aa)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:38.372418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:38.566920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:38.569247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4c516d03e24fe1872b5ae584dbb1aa'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.650282 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4c516d03e24fe1872b5ae584dbb1aa'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.662994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4c516d03e24fe1872b5ae584dbb1aa'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.669895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4c516d03e24fe1872b5ae584dbb1aa'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.670607 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-41e011e4-1b86-4a26-bbe6-19032425ea2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4c516d03e24fe1872b5ae584dbb1aa'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:38.677257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1210/2438] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:38 2022] DELETE /identity/v3/users/8b4c516d03e24fe1872b5ae584dbb1aa => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:39.103610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:39.107403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:39.122134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0d462-38d6-4112-b2e6-6a8a7dc2995e admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:39.203992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:39.204677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:39.204964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:39.205241 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] PATH_INFO: `/v3/projects/fbb890875f904cbdad4f7bf8afa6384e` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:39.206436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb890875f904cbdad4f7bf8afa6384e)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:39.208576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:39.384129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb890875f904cbdad4f7bf8afa6384e', 'request_id': 'req-fee0e318-77c7-4d69-83a5-6c509f626033'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:39.394258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb890875f904cbdad4f7bf8afa6384e', 'request_id': 'req-fee0e318-77c7-4d69-83a5-6c509f626033'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:39.490314 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb890875f904cbdad4f7bf8afa6384e', 'request_id': 'req-fee0e318-77c7-4d69-83a5-6c509f626033'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:39.505210 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] The token cache is being invalidate because project fbb890875f904cbdad4f7bf8afa6384e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:39.505838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:39.506754 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fee0e318-77c7-4d69-83a5-6c509f626033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:39.514807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1229/2439] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:39 2022] DELETE /identity/v3/projects/fbb890875f904cbdad4f7bf8afa6384e => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:39.519584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02af33-77fb-4937-8a26-0a3968de90bd None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:39.525064 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02af33-77fb-4937-8a26-0a3968de90bd None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:39.526040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02af33-77fb-4937-8a26-0a3968de90bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:39.553992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-af02af33-77fb-4937-8a26-0a3968de90bd None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:39.555361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-af02af33-77fb-4937-8a26-0a3968de90bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:39.634758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1211/2440] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:39.637710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe143-4871-4803-a83d-20fe21dd5aa9 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:39.638106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe143-4871-4803-a83d-20fe21dd5aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:39.638409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe143-4871-4803-a83d-20fe21dd5aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:39.661723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-881fe143-4871-4803-a83d-20fe21dd5aa9 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:39.662975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-881fe143-4871-4803-a83d-20fe21dd5aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:39.671485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1230/2441] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:39.675965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02af33-77fb-4937-8a26-0a3968de90bd None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:39.687300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02af33-77fb-4937-8a26-0a3968de90bd None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:39.692839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d299b6b-d73f-460a-a6c5-914e59a15673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:39.693462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d299b6b-d73f-460a-a6c5-914e59a15673 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:39.693745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d299b6b-d73f-460a-a6c5-914e59a15673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:39.694378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d299b6b-d73f-460a-a6c5-914e59a15673 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:39.694816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d299b6b-d73f-460a-a6c5-914e59a15673 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:59:39.695125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d299b6b-d73f-460a-a6c5-914e59a15673 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:39.696985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d299b6b-d73f-460a-a6c5-914e59a15673 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:39.712780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1212/2442] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:59:39.717625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe143-4871-4803-a83d-20fe21dd5aa9 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:39.729502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe143-4871-4803-a83d-20fe21dd5aa9 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:39.736098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee703d-72ad-4ecf-80cd-54107fa710c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:39.736738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee703d-72ad-4ecf-80cd-54107fa710c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:39.737023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee703d-72ad-4ecf-80cd-54107fa710c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:39.737307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee703d-72ad-4ecf-80cd-54107fa710c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:39.738113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee703d-72ad-4ecf-80cd-54107fa710c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:39.740166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee703d-72ad-4ecf-80cd-54107fa710c1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:39.871461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1231/2443] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:39 2022] POST /identity/v3/projects => generated 393 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:39.875040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d299b6b-d73f-460a-a6c5-914e59a15673 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:39.886665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d299b6b-d73f-460a-a6c5-914e59a15673 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:39.960898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5fa48-7ef2-4aaf-a91c-1692881f93b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:39.960898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5fa48-7ef2-4aaf-a91c-1692881f93b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:39.960898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5fa48-7ef2-4aaf-a91c-1692881f93b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:39.960898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5fa48-7ef2-4aaf-a91c-1692881f93b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:39.961324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5fa48-7ef2-4aaf-a91c-1692881f93b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:39.961324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5fa48-7ef2-4aaf-a91c-1692881f93b7 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:40.149383 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1213/2444] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:39 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:40.153983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee703d-72ad-4ecf-80cd-54107fa710c1 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:40.175374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee703d-72ad-4ecf-80cd-54107fa710c1 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:40.181777 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2ac53-9c63-45c4-9034-194625377dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:40.182673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2ac53-9c63-45c4-9034-194625377dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:40.182981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2ac53-9c63-45c4-9034-194625377dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:40.183262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2ac53-9c63-45c4-9034-194625377dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:40.183920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2ac53-9c63-45c4-9034-194625377dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:40.185722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2ac53-9c63-45c4-9034-194625377dcb admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:40.201647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1232/2445] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:40.207146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5fa48-7ef2-4aaf-a91c-1692881f93b7 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:40.220231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5fa48-7ef2-4aaf-a91c-1692881f93b7 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:40.224992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f6c31-6184-4c5f-be21-07a032cf4ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:40.225609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f6c31-6184-4c5f-be21-07a032cf4ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:40.225895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f6c31-6184-4c5f-be21-07a032cf4ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:40.226174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f6c31-6184-4c5f-be21-07a032cf4ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:40.226865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f6c31-6184-4c5f-be21-07a032cf4ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:40.228689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f6c31-6184-4c5f-be21-07a032cf4ef8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:40.242485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1214/2446] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:40.246727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2ac53-9c63-45c4-9034-194625377dcb admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:40.258609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2ac53-9c63-45c4-9034-194625377dcb admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:40.263387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d4b16-b8b3-4c75-8167-038537a0fb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:40.264058 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d4b16-b8b3-4c75-8167-038537a0fb50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:40.264983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d4b16-b8b3-4c75-8167-038537a0fb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:40.264983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d4b16-b8b3-4c75-8167-038537a0fb50 admin admin] PATH_INFO: `/v3/projects/0b7392feb4e541f3af1b42f8f4a0171c/users/f268722e08b84e899e45b4e5c9237a50/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:40.277252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:40.282265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d4b16-b8b3-4c75-8167-038537a0fb50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7392feb4e541f3af1b42f8f4a0171c, user_id=f268722e08b84e899e45b4e5c9237a50, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:40.285305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d4b16-b8b3-4c75-8167-038537a0fb50 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:40.417550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1233/2447] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:40 2022] PUT /identity/v3/projects/0b7392feb4e541f3af1b42f8f4a0171c/users/f268722e08b84e899e45b4e5c9237a50/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:40.430139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640922f5-a520-425c-aa18-2019b2500560 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:40.430440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640922f5-a520-425c-aa18-2019b2500560 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:40.430777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640922f5-a520-425c-aa18-2019b2500560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:40.469446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-640922f5-a520-425c-aa18-2019b2500560 None None] MFA Rules not processed for user `f268722e08b84e899e45b4e5c9237a50`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:40.470922 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-640922f5-a520-425c-aa18-2019b2500560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:40.489742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:40.548706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1215/2448] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:40.553738 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d4b16-b8b3-4c75-8167-038537a0fb50 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:40.640847 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d4b16-b8b3-4c75-8167-038537a0fb50 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:40.645818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e54dd-7fbf-4bae-aa26-7f8159fa5232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:40.646433 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e54dd-7fbf-4bae-aa26-7f8159fa5232 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:40.655911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e54dd-7fbf-4bae-aa26-7f8159fa5232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:40.655911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e54dd-7fbf-4bae-aa26-7f8159fa5232 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:40.655911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e54dd-7fbf-4bae-aa26-7f8159fa5232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:40.655911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e54dd-7fbf-4bae-aa26-7f8159fa5232 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:40.776990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1234/2449] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:40 2022] POST /identity/v3/projects => generated 393 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:40.781011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640922f5-a520-425c-aa18-2019b2500560 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:40.793831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640922f5-a520-425c-aa18-2019b2500560 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:40.865446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297824cf-3a5c-43db-a879-063956a2eba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:40.866136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297824cf-3a5c-43db-a879-063956a2eba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:40.866421 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297824cf-3a5c-43db-a879-063956a2eba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:40.866731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297824cf-3a5c-43db-a879-063956a2eba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:40.867409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297824cf-3a5c-43db-a879-063956a2eba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:40.869371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297824cf-3a5c-43db-a879-063956a2eba5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:40.993377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1216/2450] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:40 2022] POST /identity/v3/users => generated 303 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:40.997462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e54dd-7fbf-4bae-aa26-7f8159fa5232 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:41.009126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e54dd-7fbf-4bae-aa26-7f8159fa5232 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:41.013997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7a48c-9f4c-44f9-b8f1-03f1b7461037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:41.014913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7a48c-9f4c-44f9-b8f1-03f1b7461037 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.015193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7a48c-9f4c-44f9-b8f1-03f1b7461037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.015413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7a48c-9f4c-44f9-b8f1-03f1b7461037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.018648 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7a48c-9f4c-44f9-b8f1-03f1b7461037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:41.019043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7a48c-9f4c-44f9-b8f1-03f1b7461037 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:41.034555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1235/2451] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:41.039551 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297824cf-3a5c-43db-a879-063956a2eba5 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:41.058008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297824cf-3a5c-43db-a879-063956a2eba5 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:41.063793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371adf7-d500-4a6a-8463-e9e01afdfb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:41.064654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371adf7-d500-4a6a-8463-e9e01afdfb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.065228 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371adf7-d500-4a6a-8463-e9e01afdfb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.065653 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371adf7-d500-4a6a-8463-e9e01afdfb06 admin admin] PATH_INFO: `/v3/projects/101c2986bbc64368a789ec69673335bd/users/47f81caa8ae0446aaffe9d09a95aacb0/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.089097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371adf7-d500-4a6a-8463-e9e01afdfb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=101c2986bbc64368a789ec69673335bd, user_id=47f81caa8ae0446aaffe9d09a95aacb0, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:41.091892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371adf7-d500-4a6a-8463-e9e01afdfb06 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:41.238743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1217/2452] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:41 2022] PUT /identity/v3/projects/101c2986bbc64368a789ec69673335bd/users/47f81caa8ae0446aaffe9d09a95aacb0/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:41.243083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7a48c-9f4c-44f9-b8f1-03f1b7461037 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:41.255026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371adf7-d500-4a6a-8463-e9e01afdfb06 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:41.256978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7371adf7-d500-4a6a-8463-e9e01afdfb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:41.257232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7a48c-9f4c-44f9-b8f1-03f1b7461037 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:41.317818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580ad75-5dd0-4b60-b118-98ca53890d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:41.318350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580ad75-5dd0-4b60-b118-98ca53890d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.318708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580ad75-5dd0-4b60-b118-98ca53890d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.319012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580ad75-5dd0-4b60-b118-98ca53890d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.319667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580ad75-5dd0-4b60-b118-98ca53890d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:41.321271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580ad75-5dd0-4b60-b118-98ca53890d5a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:41.330530 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371adf7-d500-4a6a-8463-e9e01afdfb06 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:41.335734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:41.336473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1236/2453] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:41.336594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.336679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.336954 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] PATH_INFO: `/v3/users/b4dd4d33280d4284a4e0bd673ac72641` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.338243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4dd4d33280d4284a4e0bd673ac72641)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:41.340287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580ad75-5dd0-4b60-b118-98ca53890d5a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:41.340474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:41.351479 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580ad75-5dd0-4b60-b118-98ca53890d5a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:41.356611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dceee2-715d-4d4c-b301-4f1bcc817c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:41.357306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dceee2-715d-4d4c-b301-4f1bcc817c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.357593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dceee2-715d-4d4c-b301-4f1bcc817c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.366878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dceee2-715d-4d4c-b301-4f1bcc817c31 admin admin] PATH_INFO: `/v3/projects/101c2986bbc64368a789ec69673335bd/users/47f81caa8ae0446aaffe9d09a95aacb0/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.366878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dceee2-715d-4d4c-b301-4f1bcc817c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=101c2986bbc64368a789ec69673335bd, user_id=47f81caa8ae0446aaffe9d09a95aacb0, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:41.366878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dceee2-715d-4d4c-b301-4f1bcc817c31 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:41.380377 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1237/2454] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:41 2022] PUT /identity/v3/projects/101c2986bbc64368a789ec69673335bd/users/47f81caa8ae0446aaffe9d09a95aacb0/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:41.390566 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac40cc-698b-45fc-a858-6d8f59999453 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.391418 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac40cc-698b-45fc-a858-6d8f59999453 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.391852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac40cc-698b-45fc-a858-6d8f59999453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.421358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-daac40cc-698b-45fc-a858-6d8f59999453 None None] MFA Rules not processed for user `47f81caa8ae0446aaffe9d09a95aacb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:41.422779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-daac40cc-698b-45fc-a858-6d8f59999453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:41.456435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:41.484452 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:41.490686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1238/2455] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:41.502755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac40cc-698b-45fc-a858-6d8f59999453 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:41.504358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-daac40cc-698b-45fc-a858-6d8f59999453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:41.550095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4dd4d33280d4284a4e0bd673ac72641'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:41.567426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac40cc-698b-45fc-a858-6d8f59999453 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:41.571257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba22b63-7fc6-479e-a7a8-0f931d33f039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:41.571788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba22b63-7fc6-479e-a7a8-0f931d33f039 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.572139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba22b63-7fc6-479e-a7a8-0f931d33f039 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.572454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba22b63-7fc6-479e-a7a8-0f931d33f039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.581761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba22b63-7fc6-479e-a7a8-0f931d33f039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:41.583212 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba22b63-7fc6-479e-a7a8-0f931d33f039 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:41.593125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1239/2456] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:41 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:59:41.631810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4dd4d33280d4284a4e0bd673ac72641'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:41.641367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba22b63-7fc6-479e-a7a8-0f931d33f039 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:41.643162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-dba22b63-7fc6-479e-a7a8-0f931d33f039 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:41.645158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4dd4d33280d4284a4e0bd673ac72641'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:41.653293 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4dd4d33280d4284a4e0bd673ac72641'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:41.653907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-526771e7-77eb-4244-9fbf-d148dab34532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4dd4d33280d4284a4e0bd673ac72641'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:41.656307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba22b63-7fc6-479e-a7a8-0f931d33f039 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:41.661112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:41.662016 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.662257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1218/2457] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:41 2022] DELETE /identity/v3/users/b4dd4d33280d4284a4e0bd673ac72641 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:41.662348 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.662348 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.664243 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:41.666892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71e734-905a-4a9e-8d3d-e9b341f48860 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.667422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71e734-905a-4a9e-8d3d-e9b341f48860 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.667976 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71e734-905a-4a9e-8d3d-e9b341f48860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.696684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-df71e734-905a-4a9e-8d3d-e9b341f48860 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:41.698004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-df71e734-905a-4a9e-8d3d-e9b341f48860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:41.739972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:41.741498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:41.753406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1240/2458] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:59:41.768782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1219/2459] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:41.778652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:41.780120 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:41.852190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53920b7-2d46-4dad-9953-56506b9e8aee service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:41.886690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:41.913005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b67015-7e51-43df-a52c-e3443190a7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:41.913621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b67015-7e51-43df-a52c-e3443190a7ce service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:41.913904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b67015-7e51-43df-a52c-e3443190a7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:41.914178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b67015-7e51-43df-a52c-e3443190a7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:41.925015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b67015-7e51-43df-a52c-e3443190a7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:41.926494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b67015-7e51-43df-a52c-e3443190a7ce service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:41.936266 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1241/2460] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:42.538952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71e734-905a-4a9e-8d3d-e9b341f48860 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:42.540467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-df71e734-905a-4a9e-8d3d-e9b341f48860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:42.564088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71e734-905a-4a9e-8d3d-e9b341f48860 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:42.571814 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:42.572564 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:42.573046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:42.573710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] PATH_INFO: `/v3/projects/9918ac1e9007498abbdfccce46edf4cf` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:42.575900 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9918ac1e9007498abbdfccce46edf4cf)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:42.579185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:42.722646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9918ac1e9007498abbdfccce46edf4cf', 'request_id': 'req-138cac9f-0551-4650-88f5-4e38d8c815c9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:42.868803 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9918ac1e9007498abbdfccce46edf4cf', 'request_id': 'req-138cac9f-0551-4650-88f5-4e38d8c815c9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:42.876254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9918ac1e9007498abbdfccce46edf4cf', 'request_id': 'req-138cac9f-0551-4650-88f5-4e38d8c815c9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:42.887305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] The token cache is being invalidate because project 9918ac1e9007498abbdfccce46edf4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:42.887595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:42.888342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:42.889825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1220/2461] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:42 2022] DELETE /identity/v3/projects/9918ac1e9007498abbdfccce46edf4cf => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:42.894766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b67015-7e51-43df-a52c-e3443190a7ce service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:42.905162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b67015-7e51-43df-a52c-e3443190a7ce service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:42.969655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:42.970301 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:42.970615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:42.970891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] PATH_INFO: `/v3/users/5bd42a3e0eb0429e95225411a9e59a51` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:42.972021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd42a3e0eb0429e95225411a9e59a51)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:42.973948 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:43.154570 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd42a3e0eb0429e95225411a9e59a51'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:43.155230 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd42a3e0eb0429e95225411a9e59a51'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:43.238864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd42a3e0eb0429e95225411a9e59a51'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:43.246453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd42a3e0eb0429e95225411a9e59a51'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:43.259557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1113cbeb-917c-454b-b5a5-3809df1b09e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd42a3e0eb0429e95225411a9e59a51'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:43.265676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1242/2462] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:42 2022] DELETE /identity/v3/users/5bd42a3e0eb0429e95225411a9e59a51 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:43.823985 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:43.825546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:43.840671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cac9f-0551-4650-88f5-4e38d8c815c9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:43.868567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:43.902178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:43.905269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:43.905269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:43.905269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] PATH_INFO: `/v3/projects/e30bf7701f8e4b8c8277df525a9099e6` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:43.905269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e30bf7701f8e4b8c8277df525a9099e6)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:43.907302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:44.045453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e30bf7701f8e4b8c8277df525a9099e6', 'request_id': 'req-c4e150f1-cb28-4974-8a41-18d0ed995dcc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:44.141824 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e30bf7701f8e4b8c8277df525a9099e6', 'request_id': 'req-c4e150f1-cb28-4974-8a41-18d0ed995dcc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:44.157544 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e30bf7701f8e4b8c8277df525a9099e6', 'request_id': 'req-c4e150f1-cb28-4974-8a41-18d0ed995dcc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:44.174181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] The token cache is being invalidate because project e30bf7701f8e4b8c8277df525a9099e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:44.174476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:44.175216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c4e150f1-cb28-4974-8a41-18d0ed995dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:44.177050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1221/2463] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:43 2022] DELETE /identity/v3/projects/e30bf7701f8e4b8c8277df525a9099e6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:44.187111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb751b94-defa-481d-a0d0-69b9d2b51a2a None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:44.187111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb751b94-defa-481d-a0d0-69b9d2b51a2a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:44.187111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb751b94-defa-481d-a0d0-69b9d2b51a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:44.207932 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-fb751b94-defa-481d-a0d0-69b9d2b51a2a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:44.209152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fb751b94-defa-481d-a0d0-69b9d2b51a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:44.255669 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:44.270460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1243/2464] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:44.278548 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6d2cf-3a64-405b-afab-97316ba3720f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:44.278847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6d2cf-3a64-405b-afab-97316ba3720f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:44.279213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6d2cf-3a64-405b-afab-97316ba3720f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:44.300897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e2d6d2cf-3a64-405b-afab-97316ba3720f None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:44.302139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e2d6d2cf-3a64-405b-afab-97316ba3720f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:44.311081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1222/2465] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:44.316326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb751b94-defa-481d-a0d0-69b9d2b51a2a None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:44.326000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb751b94-defa-481d-a0d0-69b9d2b51a2a None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:44.331395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fd484-17fc-43a9-af3d-c11f366af133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:44.331995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fd484-17fc-43a9-af3d-c11f366af133 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:44.332277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fd484-17fc-43a9-af3d-c11f366af133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:44.332548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fd484-17fc-43a9-af3d-c11f366af133 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:44.333193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fd484-17fc-43a9-af3d-c11f366af133 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:59:44.333495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fd484-17fc-43a9-af3d-c11f366af133 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:44.335135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fd484-17fc-43a9-af3d-c11f366af133 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:44.347419 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1244/2466] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:59:44.352256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6d2cf-3a64-405b-afab-97316ba3720f None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:44.362135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6d2cf-3a64-405b-afab-97316ba3720f None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:44.368046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea093b6-addc-4944-9c05-0e868ac7499c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:44.368655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea093b6-addc-4944-9c05-0e868ac7499c admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:44.368935 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea093b6-addc-4944-9c05-0e868ac7499c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:44.369211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea093b6-addc-4944-9c05-0e868ac7499c admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:44.369844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea093b6-addc-4944-9c05-0e868ac7499c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:44.371801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea093b6-addc-4944-9c05-0e868ac7499c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:44.496620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1223/2467] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:44 2022] POST /identity/v3/projects => generated 407 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:44.498041 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fd484-17fc-43a9-af3d-c11f366af133 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:44.510413 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fd484-17fc-43a9-af3d-c11f366af133 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:44.568589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e336b-ca85-4c1c-a6f2-0e92c8353e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:44.569214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e336b-ca85-4c1c-a6f2-0e92c8353e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:44.569498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e336b-ca85-4c1c-a6f2-0e92c8353e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:44.569776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e336b-ca85-4c1c-a6f2-0e92c8353e24 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:44.570417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e336b-ca85-4c1c-a6f2-0e92c8353e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:44.572432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e336b-ca85-4c1c-a6f2-0e92c8353e24 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:44.695471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1245/2468] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:44 2022] POST /identity/v3/users => generated 304 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:44.699885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea093b6-addc-4944-9c05-0e868ac7499c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:44.710739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea093b6-addc-4944-9c05-0e868ac7499c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:44.715285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd0478-d760-4fe9-b3ea-d2b5457c0c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:44.715927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd0478-d760-4fe9-b3ea-d2b5457c0c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:44.716210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd0478-d760-4fe9-b3ea-d2b5457c0c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:44.716485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd0478-d760-4fe9-b3ea-d2b5457c0c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:44.717159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd0478-d760-4fe9-b3ea-d2b5457c0c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:44.718981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd0478-d760-4fe9-b3ea-d2b5457c0c83 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:44.728875 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1224/2469] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:44.732858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e336b-ca85-4c1c-a6f2-0e92c8353e24 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:44.743712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e336b-ca85-4c1c-a6f2-0e92c8353e24 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:44.748444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c49a36-c092-4cf7-acb1-d14e3d9e23cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:44.749040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c49a36-c092-4cf7-acb1-d14e3d9e23cb admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:44.749321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c49a36-c092-4cf7-acb1-d14e3d9e23cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:44.749594 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c49a36-c092-4cf7-acb1-d14e3d9e23cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:44.750894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c49a36-c092-4cf7-acb1-d14e3d9e23cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:44.752494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c49a36-c092-4cf7-acb1-d14e3d9e23cb admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:44.763462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1246/2470] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:44.773537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd0478-d760-4fe9-b3ea-d2b5457c0c83 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:44.783625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd0478-d760-4fe9-b3ea-d2b5457c0c83 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:44.790639 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ad679-cd1f-4615-9513-859feb8c7aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:44.791309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ad679-cd1f-4615-9513-859feb8c7aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:44.791744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ad679-cd1f-4615-9513-859feb8c7aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:44.792043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ad679-cd1f-4615-9513-859feb8c7aa8 admin admin] PATH_INFO: `/v3/projects/c76aa917711244d6bd26a682ecb19cb7/users/379fb94d81a94fc79cfaa50efc939196/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:44.809818 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ad679-cd1f-4615-9513-859feb8c7aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76aa917711244d6bd26a682ecb19cb7, user_id=379fb94d81a94fc79cfaa50efc939196, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:44.812995 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ad679-cd1f-4615-9513-859feb8c7aa8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:44.905149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1225/2471] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:44 2022] PUT /identity/v3/projects/c76aa917711244d6bd26a682ecb19cb7/users/379fb94d81a94fc79cfaa50efc939196/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:44.919324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fcec6-1baa-4f74-9791-074e1f3e8749 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:44.919979 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fcec6-1baa-4f74-9791-074e1f3e8749 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:44.920260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fcec6-1baa-4f74-9791-074e1f3e8749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:44.950109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-8b7fcec6-1baa-4f74-9791-074e1f3e8749 None None] MFA Rules not processed for user `379fb94d81a94fc79cfaa50efc939196`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:44.951383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8b7fcec6-1baa-4f74-9791-074e1f3e8749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:45.022029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1247/2472] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:44 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:45.026338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ad679-cd1f-4615-9513-859feb8c7aa8 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:45.081774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:45.092136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ad679-cd1f-4615-9513-859feb8c7aa8 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:45.097794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6c0f8-c83d-48b9-83c2-8fc3d7511eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:45.098419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c0f8-c83d-48b9-83c2-8fc3d7511eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:45.098729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c0f8-c83d-48b9-83c2-8fc3d7511eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:45.099005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6c0f8-c83d-48b9-83c2-8fc3d7511eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:45.099654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6c0f8-c83d-48b9-83c2-8fc3d7511eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:45.101568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6c0f8-c83d-48b9-83c2-8fc3d7511eb0 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:45.228505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1226/2473] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:45 2022] POST /identity/v3/projects => generated 409 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:45.232508 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fcec6-1baa-4f74-9791-074e1f3e8749 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:45.243845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fcec6-1baa-4f74-9791-074e1f3e8749 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:45.289498 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:45.301892 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03ecce-b396-401f-9c72-e41fc57d76e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:45.302530 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ecce-b396-401f-9c72-e41fc57d76e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:45.302807 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ecce-b396-401f-9c72-e41fc57d76e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:45.303081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ecce-b396-401f-9c72-e41fc57d76e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:45.303708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03ecce-b396-401f-9c72-e41fc57d76e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:45.305643 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03ecce-b396-401f-9c72-e41fc57d76e2 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:45.473620 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1248/2474] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:45 2022] POST /identity/v3/users => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:45.477691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6c0f8-c83d-48b9-83c2-8fc3d7511eb0 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:45.488122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6c0f8-c83d-48b9-83c2-8fc3d7511eb0 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:45.492706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e36659-2e55-448b-bb79-b7a2314fa332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:45.493305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e36659-2e55-448b-bb79-b7a2314fa332 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:45.493587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e36659-2e55-448b-bb79-b7a2314fa332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:45.493863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e36659-2e55-448b-bb79-b7a2314fa332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:45.494521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e36659-2e55-448b-bb79-b7a2314fa332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:45.496148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e36659-2e55-448b-bb79-b7a2314fa332 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:45.506425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1227/2475] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:45.510934 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03ecce-b396-401f-9c72-e41fc57d76e2 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:45.522404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03ecce-b396-401f-9c72-e41fc57d76e2 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:45.527177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9198b-7841-42d9-adc2-979985da6165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:45.527824 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9198b-7841-42d9-adc2-979985da6165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:45.528103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9198b-7841-42d9-adc2-979985da6165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:45.528379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9198b-7841-42d9-adc2-979985da6165 admin admin] PATH_INFO: `/v3/projects/4298378c160f4e4785d8794e82217dfe/users/1b7f12360c6b4a38a3219dff3059a4e9/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:45.545737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9198b-7841-42d9-adc2-979985da6165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4298378c160f4e4785d8794e82217dfe, user_id=1b7f12360c6b4a38a3219dff3059a4e9, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:45.548508 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9198b-7841-42d9-adc2-979985da6165 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:45.619403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e36659-2e55-448b-bb79-b7a2314fa332 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:45.621192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d3e36659-2e55-448b-bb79-b7a2314fa332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:45.663231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1249/2476] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:45 2022] PUT /identity/v3/projects/4298378c160f4e4785d8794e82217dfe/users/1b7f12360c6b4a38a3219dff3059a4e9/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:45.667315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9198b-7841-42d9-adc2-979985da6165 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:45.681899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e36659-2e55-448b-bb79-b7a2314fa332 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:45.734830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df9f27-7be9-4cea-ae17-3c7764fc511b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:45.735400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df9f27-7be9-4cea-ae17-3c7764fc511b service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:45.735673 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df9f27-7be9-4cea-ae17-3c7764fc511b service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:45.735948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df9f27-7be9-4cea-ae17-3c7764fc511b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:45.736400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9198b-7841-42d9-adc2-979985da6165 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:45.737709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a3df9f27-7be9-4cea-ae17-3c7764fc511b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:45.741162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d1871-28b7-4c7b-8246-b8116e8088ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:45.744891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d1871-28b7-4c7b-8246-b8116e8088ee admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:45.744891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d1871-28b7-4c7b-8246-b8116e8088ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:45.744891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d1871-28b7-4c7b-8246-b8116e8088ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:45.744891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d1871-28b7-4c7b-8246-b8116e8088ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:45.744891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d1871-28b7-4c7b-8246-b8116e8088ee admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:45.750896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df9f27-7be9-4cea-ae17-3c7764fc511b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:45.752644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df9f27-7be9-4cea-ae17-3c7764fc511b service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:45.756736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1250/2477] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:45.760782 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d1871-28b7-4c7b-8246-b8116e8088ee admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:45.766789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1228/2478] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:59:45.771698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d1871-28b7-4c7b-8246-b8116e8088ee admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:45.776487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad948f69-8716-4917-a826-ebac6f9878b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:45.777148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad948f69-8716-4917-a826-ebac6f9878b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:45.777486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad948f69-8716-4917-a826-ebac6f9878b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:45.777769 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad948f69-8716-4917-a826-ebac6f9878b8 admin admin] PATH_INFO: `/v3/projects/4298378c160f4e4785d8794e82217dfe/users/1b7f12360c6b4a38a3219dff3059a4e9/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:45.779863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad948f69-8716-4917-a826-ebac6f9878b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4298378c160f4e4785d8794e82217dfe, user_id=1b7f12360c6b4a38a3219dff3059a4e9, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:45.783239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad948f69-8716-4917-a826-ebac6f9878b8 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:45.794151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1251/2479] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:45 2022] PUT /identity/v3/projects/4298378c160f4e4785d8794e82217dfe/users/1b7f12360c6b4a38a3219dff3059a4e9/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:45.805855 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb909a-ad9c-4281-b8c5-a783e62c834e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:45.806139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb909a-ad9c-4281-b8c5-a783e62c834e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:45.806418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb909a-ad9c-4281-b8c5-a783e62c834e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:45.836020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-91bb909a-ad9c-4281-b8c5-a783e62c834e None None] MFA Rules not processed for user `1b7f12360c6b4a38a3219dff3059a4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:45.837308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-91bb909a-ad9c-4281-b8c5-a783e62c834e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:45.917089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1229/2480] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:45 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:45.931018 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad948f69-8716-4917-a826-ebac6f9878b8 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:45.932998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ad948f69-8716-4917-a826-ebac6f9878b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:46.000187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad948f69-8716-4917-a826-ebac6f9878b8 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:46.004593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248ce55-c6e1-45e7-8c0a-bb0a9d50cd20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:46.005265 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248ce55-c6e1-45e7-8c0a-bb0a9d50cd20 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:46.005558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248ce55-c6e1-45e7-8c0a-bb0a9d50cd20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:46.005834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248ce55-c6e1-45e7-8c0a-bb0a9d50cd20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:46.016821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248ce55-c6e1-45e7-8c0a-bb0a9d50cd20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:46.018899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248ce55-c6e1-45e7-8c0a-bb0a9d50cd20 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:46.029598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1252/2481] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:45 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:46.358348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb909a-ad9c-4281-b8c5-a783e62c834e None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:46.359860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-91bb909a-ad9c-4281-b8c5-a783e62c834e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:46.372023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb909a-ad9c-4281-b8c5-a783e62c834e None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:46.387268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c7d4b-b7a2-4a0c-a387-839ffd0d1227 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:46.391159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c7d4b-b7a2-4a0c-a387-839ffd0d1227 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:46.391159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c7d4b-b7a2-4a0c-a387-839ffd0d1227 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:46.391159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c7d4b-b7a2-4a0c-a387-839ffd0d1227 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:46.398964 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:46.450099 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c7d4b-b7a2-4a0c-a387-839ffd0d1227 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:46.451714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c7d4b-b7a2-4a0c-a387-839ffd0d1227 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:46.462865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1230/2482] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:46 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:46.953929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248ce55-c6e1-45e7-8c0a-bb0a9d50cd20 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:46.955416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5248ce55-c6e1-45e7-8c0a-bb0a9d50cd20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:47.009984 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:47.039297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248ce55-c6e1-45e7-8c0a-bb0a9d50cd20 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:47.044734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:47.045576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:47.045911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:47.047002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] PATH_INFO: `/v3/users/f268722e08b84e899e45b4e5c9237a50` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:47.048492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f268722e08b84e899e45b4e5c9237a50)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:47.050416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:47.289006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f268722e08b84e899e45b4e5c9237a50'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:47.289692 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f268722e08b84e899e45b4e5c9237a50'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:47.375947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f268722e08b84e899e45b4e5c9237a50'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:47.385788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f268722e08b84e899e45b4e5c9237a50'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:47.401662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f268722e08b84e899e45b4e5c9237a50'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:47.409121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1253/2483] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:46 2022] DELETE /identity/v3/users/f268722e08b84e899e45b4e5c9237a50 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:47.430785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ab64-f8ac-4858-9f79-752cd1c0f723 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:47.431335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ab64-f8ac-4858-9f79-752cd1c0f723 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:47.431335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ab64-f8ac-4858-9f79-752cd1c0f723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:47.451396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-72a5ab64-f8ac-4858-9f79-752cd1c0f723 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:47.461750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-72a5ab64-f8ac-4858-9f79-752cd1c0f723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:47.517463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1231/2484] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:47.532810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:47.535169 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:47.618896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb56b7-0f60-44dc-a9cf-55fc082fff2d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:47.622450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fe451-3b5f-4058-a73d-71ea34c406f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:47.623143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fe451-3b5f-4058-a73d-71ea34c406f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:47.623420 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fe451-3b5f-4058-a73d-71ea34c406f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:47.623868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fe451-3b5f-4058-a73d-71ea34c406f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:47.634141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fe451-3b5f-4058-a73d-71ea34c406f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:47.635612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fe451-3b5f-4058-a73d-71ea34c406f2 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:47.645952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1254/2485] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:48.094840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5ab64-f8ac-4858-9f79-752cd1c0f723 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:48.097133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-72a5ab64-f8ac-4858-9f79-752cd1c0f723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:48.119195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5ab64-f8ac-4858-9f79-752cd1c0f723 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:48.127671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:48.129361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:48.129857 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:48.130348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] PATH_INFO: `/v3/projects/0b7392feb4e541f3af1b42f8f4a0171c` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:48.132344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7392feb4e541f3af1b42f8f4a0171c)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:48.136175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:48.320912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7392feb4e541f3af1b42f8f4a0171c', 'request_id': 'req-bbc75480-de90-49c7-98aa-873076470075'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.387699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7392feb4e541f3af1b42f8f4a0171c', 'request_id': 'req-bbc75480-de90-49c7-98aa-873076470075'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.397656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7392feb4e541f3af1b42f8f4a0171c', 'request_id': 'req-bbc75480-de90-49c7-98aa-873076470075'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.411662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] The token cache is being invalidate because project 0b7392feb4e541f3af1b42f8f4a0171c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:48.412246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.412894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.414714 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1232/2486] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:48 2022] DELETE /identity/v3/projects/0b7392feb4e541f3af1b42f8f4a0171c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:48.418775 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fe451-3b5f-4058-a73d-71ea34c406f2 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:48.432446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fe451-3b5f-4058-a73d-71ea34c406f2 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:48.513884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:48.516409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:48.518003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:48.518552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] PATH_INFO: `/v3/users/47f81caa8ae0446aaffe9d09a95aacb0` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:48.520868 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f81caa8ae0446aaffe9d09a95aacb0)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:48.525759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:48.529015 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:48.787758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f81caa8ae0446aaffe9d09a95aacb0'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.788481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f81caa8ae0446aaffe9d09a95aacb0'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.877578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f81caa8ae0446aaffe9d09a95aacb0'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.903367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f81caa8ae0446aaffe9d09a95aacb0'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.930947 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6856e8d0-2c53-49ff-bcb6-7c4e288961f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f81caa8ae0446aaffe9d09a95aacb0'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:48.941253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1255/2487] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:48 2022] DELETE /identity/v3/users/47f81caa8ae0446aaffe9d09a95aacb0 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:49.411917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:49.415149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:49.434089 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc75480-de90-49c7-98aa-873076470075 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:49.490093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:49.530417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:49.532384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:49.532773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:49.533156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] PATH_INFO: `/v3/projects/101c2986bbc64368a789ec69673335bd` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:49.535408 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=101c2986bbc64368a789ec69673335bd)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:49.539050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:49.681263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '101c2986bbc64368a789ec69673335bd', 'request_id': 'req-f678a9de-7ff7-4c16-a069-41de397e3d03'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:49.838241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '101c2986bbc64368a789ec69673335bd', 'request_id': 'req-f678a9de-7ff7-4c16-a069-41de397e3d03'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:49.847664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '101c2986bbc64368a789ec69673335bd', 'request_id': 'req-f678a9de-7ff7-4c16-a069-41de397e3d03'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:49.860461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] The token cache is being invalidate because project 101c2986bbc64368a789ec69673335bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:49.860753 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:49.861425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f678a9de-7ff7-4c16-a069-41de397e3d03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:49.863482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1233/2488] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:49 2022] DELETE /identity/v3/projects/101c2986bbc64368a789ec69673335bd => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:49.871140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa05c0-767e-461a-a84c-11b04d0f0c74 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:49.871708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa05c0-767e-461a-a84c-11b04d0f0c74 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:49.871708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa05c0-767e-461a-a84c-11b04d0f0c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:49.898366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-25aa05c0-767e-461a-a84c-11b04d0f0c74 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:49.899971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-25aa05c0-767e-461a-a84c-11b04d0f0c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:49.980112 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1256/2489] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:49.981767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f22f-116f-4bde-aa62-f5c4db8ac7dd None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:49.982058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f22f-116f-4bde-aa62-f5c4db8ac7dd None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:49.982335 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f22f-116f-4bde-aa62-f5c4db8ac7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.009710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-3336f22f-116f-4bde-aa62-f5c4db8ac7dd None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:50.011041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3336f22f-116f-4bde-aa62-f5c4db8ac7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:50.024411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1234/2490] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:50.028914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa05c0-767e-461a-a84c-11b04d0f0c74 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:50.039031 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa05c0-767e-461a-a84c-11b04d0f0c74 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:50.045233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ee19a-118b-4e11-acb1-c9a51b323d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:50.045233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ee19a-118b-4e11-acb1-c9a51b323d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:50.045233 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ee19a-118b-4e11-acb1-c9a51b323d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:50.045611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ee19a-118b-4e11-acb1-c9a51b323d2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.046093 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ee19a-118b-4e11-acb1-c9a51b323d2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:59:50.047516 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ee19a-118b-4e11-acb1-c9a51b323d2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:50.048693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ee19a-118b-4e11-acb1-c9a51b323d2f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:50.061423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1257/2491] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:59:50.066796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336f22f-116f-4bde-aa62-f5c4db8ac7dd None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:50.076350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336f22f-116f-4bde-aa62-f5c4db8ac7dd None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:50.081758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd6c04-c69f-440c-9f89-bdd52e20f1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:50.082423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd6c04-c69f-440c-9f89-bdd52e20f1ab admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:50.082761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd6c04-c69f-440c-9f89-bdd52e20f1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:50.083744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd6c04-c69f-440c-9f89-bdd52e20f1ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.084059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd6c04-c69f-440c-9f89-bdd52e20f1ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:50.086029 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd6c04-c69f-440c-9f89-bdd52e20f1ab admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:50.201603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1235/2492] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:50 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:50.206901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ee19a-118b-4e11-acb1-c9a51b323d2f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:50.220060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ee19a-118b-4e11-acb1-c9a51b323d2f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:50.285252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f27bd-ad1c-41b4-a8f3-b3c858887471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:50.285786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f27bd-ad1c-41b4-a8f3-b3c858887471 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:50.286069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f27bd-ad1c-41b4-a8f3-b3c858887471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:50.286344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f27bd-ad1c-41b4-a8f3-b3c858887471 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.287020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f27bd-ad1c-41b4-a8f3-b3c858887471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:50.288980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f27bd-ad1c-41b4-a8f3-b3c858887471 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:50.302333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd6c04-c69f-440c-9f89-bdd52e20f1ab admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:50.304164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-79dd6c04-c69f-440c-9f89-bdd52e20f1ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:50.306318 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:50.318539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd6c04-c69f-440c-9f89-bdd52e20f1ab admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:50.323473 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:50.327063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:50.327063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:50.327063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] PATH_INFO: `/v3/users/87a0f4d8b8c341ed85618e70adb2def4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.327063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a0f4d8b8c341ed85618e70adb2def4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:50.328501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:50.473451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1258/2493] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:50 2022] POST /identity/v3/users => generated 292 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:50.477565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f27bd-ad1c-41b4-a8f3-b3c858887471 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:50.487860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f27bd-ad1c-41b4-a8f3-b3c858887471 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:50.493172 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7541a-3fb8-45d8-9bd4-dd4957fea2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:50.493817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7541a-3fb8-45d8-9bd4-dd4957fea2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:50.494100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7541a-3fb8-45d8-9bd4-dd4957fea2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:50.494379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7541a-3fb8-45d8-9bd4-dd4957fea2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.495072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7541a-3fb8-45d8-9bd4-dd4957fea2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:50.498140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7541a-3fb8-45d8-9bd4-dd4957fea2b9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:50.509045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1259/2494] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:50.513352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7541a-3fb8-45d8-9bd4-dd4957fea2b9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:50.528672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7541a-3fb8-45d8-9bd4-dd4957fea2b9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:50.534141 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782b871-9e68-48ab-94ac-03ea180430ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:50.534800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b871-9e68-48ab-94ac-03ea180430ae admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:50.535079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b871-9e68-48ab-94ac-03ea180430ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:50.535352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b871-9e68-48ab-94ac-03ea180430ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.536004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782b871-9e68-48ab-94ac-03ea180430ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:50.537616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782b871-9e68-48ab-94ac-03ea180430ae admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:50.548706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1260/2495] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:50.553046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782b871-9e68-48ab-94ac-03ea180430ae admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:50.563937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782b871-9e68-48ab-94ac-03ea180430ae admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:50.568613 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c951e-6bbe-4433-91e5-ab3d18a07c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:50.569273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c951e-6bbe-4433-91e5-ab3d18a07c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:50.569559 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c951e-6bbe-4433-91e5-ab3d18a07c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:50.569838 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c951e-6bbe-4433-91e5-ab3d18a07c2c admin admin] PATH_INFO: `/v3/projects/74303533ca984a5e921512122dd24951/users/3574f74ddc0146089d7771df96ff34ae/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.586571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c951e-6bbe-4433-91e5-ab3d18a07c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=74303533ca984a5e921512122dd24951, user_id=3574f74ddc0146089d7771df96ff34ae, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:50.589296 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c951e-6bbe-4433-91e5-ab3d18a07c2c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:50.591168 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a0f4d8b8c341ed85618e70adb2def4'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:50.684292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a0f4d8b8c341ed85618e70adb2def4'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:50.692144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1261/2496] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:50 2022] PUT /identity/v3/projects/74303533ca984a5e921512122dd24951/users/3574f74ddc0146089d7771df96ff34ae/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:50.703084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9337a7f-7dbf-4dfe-b457-49e3f82057b0 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:50.703250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a0f4d8b8c341ed85618e70adb2def4'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:50.703368 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9337a7f-7dbf-4dfe-b457-49e3f82057b0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:50.703641 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9337a7f-7dbf-4dfe-b457-49e3f82057b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.710361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a0f4d8b8c341ed85618e70adb2def4'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:50.711129 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a0f4d8b8c341ed85618e70adb2def4'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:50.717621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1236/2497] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:50 2022] DELETE /identity/v3/users/87a0f4d8b8c341ed85618e70adb2def4 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:50.732737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b9337a7f-7dbf-4dfe-b457-49e3f82057b0 None None] MFA Rules not processed for user `3574f74ddc0146089d7771df96ff34ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:50.734484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b9337a7f-7dbf-4dfe-b457-49e3f82057b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:50.805201 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1262/2498] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:50.809877 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:50.811320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:50.822138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:50.881010 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5427ae-f3a0-454a-bc53-69a1c0b3465e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:50.885805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa7060-fe75-4bb9-b136-0393860e2b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:50.886414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa7060-fe75-4bb9-b136-0393860e2b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:50.886722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa7060-fe75-4bb9-b136-0393860e2b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:50.886994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa7060-fe75-4bb9-b136-0393860e2b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:50.887613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa7060-fe75-4bb9-b136-0393860e2b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:50.889522 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa7060-fe75-4bb9-b136-0393860e2b05 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:50.978216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9337a7f-7dbf-4dfe-b457-49e3f82057b0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:50.981536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b9337a7f-7dbf-4dfe-b457-49e3f82057b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:50.997502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9337a7f-7dbf-4dfe-b457-49e3f82057b0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.012170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.012831 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.013116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.013484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] PATH_INFO: `/v3/projects/feac5069d37149628113087d9d93912b` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.014898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feac5069d37149628113087d9d93912b)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.016606 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.016768 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1237/2499] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:50 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:51.022130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa7060-fe75-4bb9-b136-0393860e2b05 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.085441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa7060-fe75-4bb9-b136-0393860e2b05 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.090238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc4a4d-84f9-4173-8c96-51cff054480f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.090885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc4a4d-84f9-4173-8c96-51cff054480f admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.091398 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc4a4d-84f9-4173-8c96-51cff054480f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.091683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc4a4d-84f9-4173-8c96-51cff054480f admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.092055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc4a4d-84f9-4173-8c96-51cff054480f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.094761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc4a4d-84f9-4173-8c96-51cff054480f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.130281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feac5069d37149628113087d9d93912b', 'request_id': 'req-4b81192a-9b83-43c7-8153-d9e88f134f05'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:51.135788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feac5069d37149628113087d9d93912b', 'request_id': 'req-4b81192a-9b83-43c7-8153-d9e88f134f05'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:51.209602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1238/2500] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:51 2022] POST /identity/v3/users => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:51.213709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc4a4d-84f9-4173-8c96-51cff054480f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.214877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feac5069d37149628113087d9d93912b', 'request_id': 'req-4b81192a-9b83-43c7-8153-d9e88f134f05'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:51.224781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc4a4d-84f9-4173-8c96-51cff054480f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.228650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] The token cache is being invalidate because project feac5069d37149628113087d9d93912b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:51.228946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:51.229614 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4b81192a-9b83-43c7-8153-d9e88f134f05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:51.231157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1263/2501] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:50 2022] DELETE /identity/v3/projects/feac5069d37149628113087d9d93912b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:51.238434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999362-62df-4e4b-98c2-83c1f8d9f731 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.238696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999362-62df-4e4b-98c2-83c1f8d9f731 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.238982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999362-62df-4e4b-98c2-83c1f8d9f731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.258786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-84999362-62df-4e4b-98c2-83c1f8d9f731 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:51.260103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-84999362-62df-4e4b-98c2-83c1f8d9f731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:51.278684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:51.287385 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba592d-7533-4553-b40c-3bc88cc531cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.288003 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba592d-7533-4553-b40c-3bc88cc531cb admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.288292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba592d-7533-4553-b40c-3bc88cc531cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.288573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba592d-7533-4553-b40c-3bc88cc531cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.289232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba592d-7533-4553-b40c-3bc88cc531cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.290898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba592d-7533-4553-b40c-3bc88cc531cb admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.304899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1239/2502] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:51.305946 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba592d-7533-4553-b40c-3bc88cc531cb admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.307652 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5cba592d-7533-4553-b40c-3bc88cc531cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:51.320415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba592d-7533-4553-b40c-3bc88cc531cb admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.321528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1264/2503] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:51.325175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9339-442a-455a-b308-325ead03b3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.325830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9339-442a-455a-b308-325ead03b3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.326117 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9339-442a-455a-b308-325ead03b3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.326399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9339-442a-455a-b308-325ead03b3f3 admin admin] PATH_INFO: `/v3/projects/97f264b2bb9b4bdc92e93e07a7ad4d2d/users/f4dd76867e414dc8a7a8cb69759229ec/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.330246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4660d-c3f5-4e71-aa7b-b15a2fdd2b62 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.330623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4660d-c3f5-4e71-aa7b-b15a2fdd2b62 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.330900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4660d-c3f5-4e71-aa7b-b15a2fdd2b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.342968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb9339-442a-455a-b308-325ead03b3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f264b2bb9b4bdc92e93e07a7ad4d2d, user_id=f4dd76867e414dc8a7a8cb69759229ec, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.346590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb9339-442a-455a-b308-325ead03b3f3 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.348899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:51.350805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d1e4660d-c3f5-4e71-aa7b-b15a2fdd2b62 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:51.352012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d1e4660d-c3f5-4e71-aa7b-b15a2fdd2b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:51.359873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1265/2504] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:51.364317 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4660d-c3f5-4e71-aa7b-b15a2fdd2b62 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.373877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4660d-c3f5-4e71-aa7b-b15a2fdd2b62 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.377759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c75912-2750-45cf-afd7-827d0b3b9746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.378665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c75912-2750-45cf-afd7-827d0b3b9746 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.378942 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c75912-2750-45cf-afd7-827d0b3b9746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.379261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c75912-2750-45cf-afd7-827d0b3b9746 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.379889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c75912-2750-45cf-afd7-827d0b3b9746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:59:51.380188 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c75912-2750-45cf-afd7-827d0b3b9746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.381759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c75912-2750-45cf-afd7-827d0b3b9746 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.393950 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1266/2505] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 03:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:59:51.398093 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c75912-2750-45cf-afd7-827d0b3b9746 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.407916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c75912-2750-45cf-afd7-827d0b3b9746 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.412440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6e16e-83d7-40df-bdb6-ff9262fed06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.413003 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6e16e-83d7-40df-bdb6-ff9262fed06f admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.413278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6e16e-83d7-40df-bdb6-ff9262fed06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.413600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6e16e-83d7-40df-bdb6-ff9262fed06f admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.414205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6e16e-83d7-40df-bdb6-ff9262fed06f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.416166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6e16e-83d7-40df-bdb6-ff9262fed06f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.467337 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1240/2506] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:51 2022] PUT /identity/v3/projects/97f264b2bb9b4bdc92e93e07a7ad4d2d/users/f4dd76867e414dc8a7a8cb69759229ec/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:51.475296 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9339-442a-455a-b308-325ead03b3f3 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.535677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1267/2507] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:51 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:51.536781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6e16e-83d7-40df-bdb6-ff9262fed06f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.539257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9339-442a-455a-b308-325ead03b3f3 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.609409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a67b8-bcf0-44cf-b913-321c28b2cd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.610638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a67b8-bcf0-44cf-b913-321c28b2cd47 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.610638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a67b8-bcf0-44cf-b913-321c28b2cd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.611266 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a67b8-bcf0-44cf-b913-321c28b2cd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.611579 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a67b8-bcf0-44cf-b913-321c28b2cd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.613208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a67b8-bcf0-44cf-b913-321c28b2cd47 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.615820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6e16e-83d7-40df-bdb6-ff9262fed06f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.621100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10c23c-be83-4e81-8584-13ebbb81ad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.621720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10c23c-be83-4e81-8584-13ebbb81ad5d admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.622000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10c23c-be83-4e81-8584-13ebbb81ad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.622274 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10c23c-be83-4e81-8584-13ebbb81ad5d admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.622957 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10c23c-be83-4e81-8584-13ebbb81ad5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.624103 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1241/2508] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:51.628216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10c23c-be83-4e81-8584-13ebbb81ad5d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.629237 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a67b8-bcf0-44cf-b913-321c28b2cd47 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.642767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a67b8-bcf0-44cf-b913-321c28b2cd47 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.647581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504aef73-79cf-4246-b39d-050fd26099c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.648336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aef73-79cf-4246-b39d-050fd26099c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.656965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aef73-79cf-4246-b39d-050fd26099c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.656965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aef73-79cf-4246-b39d-050fd26099c1 admin admin] PATH_INFO: `/v3/projects/97f264b2bb9b4bdc92e93e07a7ad4d2d/users/f4dd76867e414dc8a7a8cb69759229ec/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.656965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504aef73-79cf-4246-b39d-050fd26099c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f264b2bb9b4bdc92e93e07a7ad4d2d, user_id=f4dd76867e414dc8a7a8cb69759229ec, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.656965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504aef73-79cf-4246-b39d-050fd26099c1 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.664721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1242/2509] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:51 2022] PUT /identity/v3/projects/97f264b2bb9b4bdc92e93e07a7ad4d2d/users/f4dd76867e414dc8a7a8cb69759229ec/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:51.676459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56c51b-0566-4aaa-9ec0-7a4c8cde9d22 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.676743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56c51b-0566-4aaa-9ec0-7a4c8cde9d22 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.677018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56c51b-0566-4aaa-9ec0-7a4c8cde9d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.704868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-ae56c51b-0566-4aaa-9ec0-7a4c8cde9d22 None None] MFA Rules not processed for user `f4dd76867e414dc8a7a8cb69759229ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:51.706111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ae56c51b-0566-4aaa-9ec0-7a4c8cde9d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:51.737653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1268/2510] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:51 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:59:51.742560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10c23c-be83-4e81-8584-13ebbb81ad5d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.753510 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10c23c-be83-4e81-8584-13ebbb81ad5d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.782169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1243/2511] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:51.793371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56c51b-0566-4aaa-9ec0-7a4c8cde9d22 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.794901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ae56c51b-0566-4aaa-9ec0-7a4c8cde9d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:51.807404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:51.821603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585325b-fcc8-4542-b279-77674cb55911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.822217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585325b-fcc8-4542-b279-77674cb55911 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.822517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585325b-fcc8-4542-b279-77674cb55911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.822799 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585325b-fcc8-4542-b279-77674cb55911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.823442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585325b-fcc8-4542-b279-77674cb55911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.825046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585325b-fcc8-4542-b279-77674cb55911 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.838210 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1269/2512] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:51.839259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585325b-fcc8-4542-b279-77674cb55911 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.849971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585325b-fcc8-4542-b279-77674cb55911 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.854678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32fd56-1134-48b3-b707-0a7f1a9d1fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.855261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56c51b-0566-4aaa-9ec0-7a4c8cde9d22 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.855387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32fd56-1134-48b3-b707-0a7f1a9d1fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.855539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32fd56-1134-48b3-b707-0a7f1a9d1fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.855812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32fd56-1134-48b3-b707-0a7f1a9d1fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.856449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32fd56-1134-48b3-b707-0a7f1a9d1fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.858028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32fd56-1134-48b3-b707-0a7f1a9d1fcc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.859678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653c1f3-e771-4a60-9171-b94f0b90553d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.860290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653c1f3-e771-4a60-9171-b94f0b90553d service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.860572 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653c1f3-e771-4a60-9171-b94f0b90553d service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.861144 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653c1f3-e771-4a60-9171-b94f0b90553d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.861256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:51.870131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653c1f3-e771-4a60-9171-b94f0b90553d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.873930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1270/2513] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:59:51.874063 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653c1f3-e771-4a60-9171-b94f0b90553d service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:51.875131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32fd56-1134-48b3-b707-0a7f1a9d1fcc admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:51.886691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1244/2514] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:51.892284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32fd56-1134-48b3-b707-0a7f1a9d1fcc admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:51.899493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989315d-a5b7-48b5-8801-0b961316f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:51.900343 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989315d-a5b7-48b5-8801-0b961316f163 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:51.900728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989315d-a5b7-48b5-8801-0b961316f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:51.901150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989315d-a5b7-48b5-8801-0b961316f163 admin admin] PATH_INFO: `/v3/projects/fc9d7ceca2474de39949bea6bbe0c2d0/users/62db2c233baf42b796523ca625e207c7/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:51.927583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989315d-a5b7-48b5-8801-0b961316f163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc9d7ceca2474de39949bea6bbe0c2d0, user_id=62db2c233baf42b796523ca625e207c7, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:51.932474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989315d-a5b7-48b5-8801-0b961316f163 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:52.001757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653c1f3-e771-4a60-9171-b94f0b90553d service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:52.010928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653c1f3-e771-4a60-9171-b94f0b90553d service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:52.015368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0f0ba-f10b-4a33-86d1-e9e2cf481517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:52.015986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0f0ba-f10b-4a33-86d1-e9e2cf481517 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:52.016265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0f0ba-f10b-4a33-86d1-e9e2cf481517 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:52.016535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0f0ba-f10b-4a33-86d1-e9e2cf481517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:52.018336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-56d0f0ba-f10b-4a33-86d1-e9e2cf481517 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:52.062042 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1271/2515] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:51 2022] PUT /identity/v3/projects/fc9d7ceca2474de39949bea6bbe0c2d0/users/62db2c233baf42b796523ca625e207c7/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:52.073383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141d797-1d5f-47c7-bd8f-5125d9dffb67 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:52.073679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141d797-1d5f-47c7-bd8f-5125d9dffb67 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:52.073959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141d797-1d5f-47c7-bd8f-5125d9dffb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:52.083228 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0f0ba-f10b-4a33-86d1-e9e2cf481517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:52.084848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0f0ba-f10b-4a33-86d1-e9e2cf481517 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:52.095190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1245/2516] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:59:52.103116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-3141d797-1d5f-47c7-bd8f-5125d9dffb67 None None] MFA Rules not processed for user `62db2c233baf42b796523ca625e207c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:52.104354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3141d797-1d5f-47c7-bd8f-5125d9dffb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:52.171134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1272/2517] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:52.175750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0f0ba-f10b-4a33-86d1-e9e2cf481517 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:52.186532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0f0ba-f10b-4a33-86d1-e9e2cf481517 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:52.249106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3443c5-8178-414c-8143-4974ed6716ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:52.250597 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3443c5-8178-414c-8143-4974ed6716ef admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:52.250763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3443c5-8178-414c-8143-4974ed6716ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:52.251025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3443c5-8178-414c-8143-4974ed6716ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:52.251670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3443c5-8178-414c-8143-4974ed6716ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:52.254285 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3443c5-8178-414c-8143-4974ed6716ef admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:52.361352 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1246/2518] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 03:59:52 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:52.365528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141d797-1d5f-47c7-bd8f-5125d9dffb67 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:52.377409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141d797-1d5f-47c7-bd8f-5125d9dffb67 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:52.434388 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5baed2-38fa-49fd-b176-70dc53a2757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:52.435029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5baed2-38fa-49fd-b176-70dc53a2757c admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:52.435311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5baed2-38fa-49fd-b176-70dc53a2757c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:52.435583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5baed2-38fa-49fd-b176-70dc53a2757c admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:52.436215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5baed2-38fa-49fd-b176-70dc53a2757c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:52.438391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5baed2-38fa-49fd-b176-70dc53a2757c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:52.569752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1273/2519] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 03:59:52 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:59:52.574016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3443c5-8178-414c-8143-4974ed6716ef admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:52.585663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3443c5-8178-414c-8143-4974ed6716ef admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:52.590350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3a8bd-3636-44b2-a4b7-1f2ab15d3499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:52.590999 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3a8bd-3636-44b2-a4b7-1f2ab15d3499 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:52.591283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3a8bd-3636-44b2-a4b7-1f2ab15d3499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:52.591560 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3a8bd-3636-44b2-a4b7-1f2ab15d3499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:52.592240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3a8bd-3636-44b2-a4b7-1f2ab15d3499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:52.594989 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3a8bd-3636-44b2-a4b7-1f2ab15d3499 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:52.605839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1247/2520] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:52.611603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5baed2-38fa-49fd-b176-70dc53a2757c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:52.621588 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5baed2-38fa-49fd-b176-70dc53a2757c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:52.626786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f435b-42b8-46a6-bc9f-c302dc6f9317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:52.627447 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f435b-42b8-46a6-bc9f-c302dc6f9317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:52.627727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f435b-42b8-46a6-bc9f-c302dc6f9317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:52.628268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f435b-42b8-46a6-bc9f-c302dc6f9317 admin admin] PATH_INFO: `/v3/projects/74a63f91b65c4a91b76190695f26a05c/users/c2bc2801e9d3479fa8a41465b73c3928/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:52.643670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f435b-42b8-46a6-bc9f-c302dc6f9317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a63f91b65c4a91b76190695f26a05c, user_id=c2bc2801e9d3479fa8a41465b73c3928, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:52.646363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f435b-42b8-46a6-bc9f-c302dc6f9317 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:52.742328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1274/2521] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:52 2022] PUT /identity/v3/projects/74a63f91b65c4a91b76190695f26a05c/users/c2bc2801e9d3479fa8a41465b73c3928/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:52.746646 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3a8bd-3636-44b2-a4b7-1f2ab15d3499 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:52.757923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3a8bd-3636-44b2-a4b7-1f2ab15d3499 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:52.779497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:52.816197 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7d102-c3d1-4887-87ed-701e71a26382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:52.816809 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7d102-c3d1-4887-87ed-701e71a26382 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:52.817091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7d102-c3d1-4887-87ed-701e71a26382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:52.817365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7d102-c3d1-4887-87ed-701e71a26382 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:52.818039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7d102-c3d1-4887-87ed-701e71a26382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:52.819677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7d102-c3d1-4887-87ed-701e71a26382 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:52.831217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1248/2522] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 03:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:59:52.845302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f435b-42b8-46a6-bc9f-c302dc6f9317 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:52.856199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f435b-42b8-46a6-bc9f-c302dc6f9317 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:52.861426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f51f00-0cd4-4fe0-8196-32159dbaa26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:52.862142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f51f00-0cd4-4fe0-8196-32159dbaa26f admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:52.862379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f51f00-0cd4-4fe0-8196-32159dbaa26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:52.862681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f51f00-0cd4-4fe0-8196-32159dbaa26f admin admin] PATH_INFO: `/v3/projects/74a63f91b65c4a91b76190695f26a05c/users/c2bc2801e9d3479fa8a41465b73c3928/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:52.864628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f51f00-0cd4-4fe0-8196-32159dbaa26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a63f91b65c4a91b76190695f26a05c, user_id=c2bc2801e9d3479fa8a41465b73c3928, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:52.867347 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f51f00-0cd4-4fe0-8196-32159dbaa26f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:52.878218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1275/2523] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 03:59:52 2022] PUT /identity/v3/projects/74a63f91b65c4a91b76190695f26a05c/users/c2bc2801e9d3479fa8a41465b73c3928/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:52.891108 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecce4a3-abeb-4b9c-a6d8-6ad1fb628c28 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:52.891567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecce4a3-abeb-4b9c-a6d8-6ad1fb628c28 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:52.891849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecce4a3-abeb-4b9c-a6d8-6ad1fb628c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:52.924559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-7ecce4a3-abeb-4b9c-a6d8-6ad1fb628c28 None None] MFA Rules not processed for user `c2bc2801e9d3479fa8a41465b73c3928`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:52.925832 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7ecce4a3-abeb-4b9c-a6d8-6ad1fb628c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:53.001876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1249/2524] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:53.013900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f51f00-0cd4-4fe0-8196-32159dbaa26f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:53.028144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:53.082967 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f51f00-0cd4-4fe0-8196-32159dbaa26f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:53.086683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000167a4-e766-4558-a6d6-49cc5f279a73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:53.087288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000167a4-e766-4558-a6d6-49cc5f279a73 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:53.087919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000167a4-e766-4558-a6d6-49cc5f279a73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:53.088151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000167a4-e766-4558-a6d6-49cc5f279a73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:53.098198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000167a4-e766-4558-a6d6-49cc5f279a73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:53.099776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000167a4-e766-4558-a6d6-49cc5f279a73 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:53.111045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1276/2525] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:53.333024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecce4a3-abeb-4b9c-a6d8-6ad1fb628c28 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:53.342622 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecce4a3-abeb-4b9c-a6d8-6ad1fb628c28 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:53.346893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea215efc-17d5-4b0c-8478-aa93b8c8ba3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:53.347504 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea215efc-17d5-4b0c-8478-aa93b8c8ba3d service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:53.347791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea215efc-17d5-4b0c-8478-aa93b8c8ba3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:53.348067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea215efc-17d5-4b0c-8478-aa93b8c8ba3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:53.407390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea215efc-17d5-4b0c-8478-aa93b8c8ba3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:53.408843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea215efc-17d5-4b0c-8478-aa93b8c8ba3d service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:53.417917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1250/2526] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:57.629123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000167a4-e766-4558-a6d6-49cc5f279a73 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:57.693346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000167a4-e766-4558-a6d6-49cc5f279a73 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:57.697905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:57.698660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:57.698943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:57.699218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] PATH_INFO: `/v3/users/3574f74ddc0146089d7771df96ff34ae` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:57.700355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3574f74ddc0146089d7771df96ff34ae)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:57.702275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:57.935539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3574f74ddc0146089d7771df96ff34ae'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:57.936216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3574f74ddc0146089d7771df96ff34ae'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:57.979741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea215efc-17d5-4b0c-8478-aa93b8c8ba3d service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:57.981154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ea215efc-17d5-4b0c-8478-aa93b8c8ba3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:57.996073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea215efc-17d5-4b0c-8478-aa93b8c8ba3d service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:58.001442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:58.002346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:58.002636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:58.007239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] PATH_INFO: `/v3/users/379fb94d81a94fc79cfaa50efc939196` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:58.007381 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3574f74ddc0146089d7771df96ff34ae'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.007489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=379fb94d81a94fc79cfaa50efc939196)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:58.007489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:58.012227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3574f74ddc0146089d7771df96ff34ae'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.031126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-85740f0f-55da-4637-b8e1-e92ee5eed67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3574f74ddc0146089d7771df96ff34ae'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.039810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1277/2527] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:57 2022] DELETE /identity/v3/users/3574f74ddc0146089d7771df96ff34ae => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:58.044180 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e0eb3-7940-4c56-acdc-740ba73e3228 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:58.044469 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e0eb3-7940-4c56-acdc-740ba73e3228 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:58.044747 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e0eb3-7940-4c56-acdc-740ba73e3228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:58.064685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a57e0eb3-7940-4c56-acdc-740ba73e3228 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:58.065896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a57e0eb3-7940-4c56-acdc-740ba73e3228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:58.095440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:58.124431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1278/2528] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:59:58.135473 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e0eb3-7940-4c56-acdc-740ba73e3228 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:58.136875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a57e0eb3-7940-4c56-acdc-740ba73e3228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:58.196215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e0eb3-7940-4c56-acdc-740ba73e3228 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:58.200048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c71e5-128c-4599-a8d9-f9ac55ad08a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:58.200697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c71e5-128c-4599-a8d9-f9ac55ad08a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:58.200981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c71e5-128c-4599-a8d9-f9ac55ad08a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:58.201287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c71e5-128c-4599-a8d9-f9ac55ad08a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:58.211771 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c71e5-128c-4599-a8d9-f9ac55ad08a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:58.213251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c71e5-128c-4599-a8d9-f9ac55ad08a1 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:58.217491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:58.223275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1279/2529] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:58.224410 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379fb94d81a94fc79cfaa50efc939196'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.369146 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379fb94d81a94fc79cfaa50efc939196'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.382821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379fb94d81a94fc79cfaa50efc939196'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.389517 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379fb94d81a94fc79cfaa50efc939196'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.390179 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379fb94d81a94fc79cfaa50efc939196'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.396056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1251/2530] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:57 2022] DELETE /identity/v3/users/379fb94d81a94fc79cfaa50efc939196 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:58.400452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff48b4-04b1-4ed0-9660-63dbd7bd40bc None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:58.400737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff48b4-04b1-4ed0-9660-63dbd7bd40bc None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:58.401008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff48b4-04b1-4ed0-9660-63dbd7bd40bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:58.421896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-69ff48b4-04b1-4ed0-9660-63dbd7bd40bc None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:59:58.423894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-69ff48b4-04b1-4ed0-9660-63dbd7bd40bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:58.491582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1280/2531] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 03:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:59:58.501864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:58.503746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:58.567830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a853a-a412-4119-ab63-9102af05f731 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:58.571792 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851025c-ba2c-41a4-93d0-7c95216ad044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:58.572494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851025c-ba2c-41a4-93d0-7c95216ad044 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:58.572791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851025c-ba2c-41a4-93d0-7c95216ad044 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:58.573067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851025c-ba2c-41a4-93d0-7c95216ad044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:58.582897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851025c-ba2c-41a4-93d0-7c95216ad044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:58.584345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851025c-ba2c-41a4-93d0-7c95216ad044 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:58.593932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1252/2532] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:59:58.685231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff48b4-04b1-4ed0-9660-63dbd7bd40bc None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:58.685231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-69ff48b4-04b1-4ed0-9660-63dbd7bd40bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:58.699864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff48b4-04b1-4ed0-9660-63dbd7bd40bc None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:58.706109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:58.706899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:58.714983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:58.714983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] PATH_INFO: `/v3/projects/74303533ca984a5e921512122dd24951` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:58.714983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74303533ca984a5e921512122dd24951)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:58.714983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:58.868698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74303533ca984a5e921512122dd24951', 'request_id': 'req-593074d0-9087-4d90-ace1-4ebb1b1b11c9'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.874238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74303533ca984a5e921512122dd24951', 'request_id': 'req-593074d0-9087-4d90-ace1-4ebb1b1b11c9'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:58.992267 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74303533ca984a5e921512122dd24951', 'request_id': 'req-593074d0-9087-4d90-ace1-4ebb1b1b11c9'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.006066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] The token cache is being invalidate because project 74303533ca984a5e921512122dd24951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:59.006432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.020887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.020887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1281/2533] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:58 2022] DELETE /identity/v3/projects/74303533ca984a5e921512122dd24951 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:59.021640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851025c-ba2c-41a4-93d0-7c95216ad044 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:59.023007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851025c-ba2c-41a4-93d0-7c95216ad044 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:59.080851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:59.081489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:59.081773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:59.082047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] PATH_INFO: `/v3/users/f4dd76867e414dc8a7a8cb69759229ec` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:59.083148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4dd76867e414dc8a7a8cb69759229ec)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:59.085143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:59.127730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:59.127730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:59.129679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:59.135159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593074d0-9087-4d90-ace1-4ebb1b1b11c9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:59.139985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:59.140635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:59.140929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:59.141209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] PATH_INFO: `/v3/projects/c76aa917711244d6bd26a682ecb19cb7` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:59.142434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76aa917711244d6bd26a682ecb19cb7)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:59.144402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:59.380420 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4dd76867e414dc8a7a8cb69759229ec'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.381743 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76aa917711244d6bd26a682ecb19cb7', 'request_id': 'req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.389136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76aa917711244d6bd26a682ecb19cb7', 'request_id': 'req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.489461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4dd76867e414dc8a7a8cb69759229ec'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.495440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:59.502811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4dd76867e414dc8a7a8cb69759229ec'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.509872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4dd76867e414dc8a7a8cb69759229ec'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.510569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4dd76867e414dc8a7a8cb69759229ec'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.517391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1253/2534] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:59 2022] DELETE /identity/v3/users/f4dd76867e414dc8a7a8cb69759229ec => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:59:59.580873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76aa917711244d6bd26a682ecb19cb7', 'request_id': 'req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.593783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] The token cache is being invalidate because project c76aa917711244d6bd26a682ecb19cb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:59:59.595011 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.595676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:59:59.597285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1282/2535] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 03:59:59 2022] DELETE /identity/v3/projects/c76aa917711244d6bd26a682ecb19cb7 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:59:59.601393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:59.603065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:59.670850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19660156-9a66-41c9-afbd-4623b4f29441 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:59.675576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:59.676210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:59.676494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:59.676769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] PATH_INFO: `/v3/users/1b7f12360c6b4a38a3219dff3059a4e9` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:59.677894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b7f12360c6b4a38a3219dff3059a4e9)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:59.679885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:59.989320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b7f12360c6b4a38a3219dff3059a4e9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.051579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:00.054907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:00.077397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b7f12360c6b4a38a3219dff3059a4e9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.091996 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b7f12360c6b4a38a3219dff3059a4e9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.099300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b7f12360c6b4a38a3219dff3059a4e9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.100201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f9cd6065-f3ac-4c39-8dad-5951e2e249a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b7f12360c6b4a38a3219dff3059a4e9'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.106534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1254/2536] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 03:59:59 2022] DELETE /identity/v3/users/1b7f12360c6b4a38a3219dff3059a4e9 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:00.121802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8aea0-e45e-4ae6-ae8a-89bca6170cc2 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:00.181326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:00.181972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.182252 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.182722 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] PATH_INFO: `/v3/projects/97f264b2bb9b4bdc92e93e07a7ad4d2d` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.183888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f264b2bb9b4bdc92e93e07a7ad4d2d)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:00.185842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:00.328320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:00.340245 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f264b2bb9b4bdc92e93e07a7ad4d2d', 'request_id': 'req-fc67fb2c-3cfa-428f-8916-50c7107efae6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.350752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f264b2bb9b4bdc92e93e07a7ad4d2d', 'request_id': 'req-fc67fb2c-3cfa-428f-8916-50c7107efae6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.448464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f264b2bb9b4bdc92e93e07a7ad4d2d', 'request_id': 'req-fc67fb2c-3cfa-428f-8916-50c7107efae6'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.461888 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] The token cache is being invalidate because project 97f264b2bb9b4bdc92e93e07a7ad4d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:00.462260 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.462952 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-fc67fb2c-3cfa-428f-8916-50c7107efae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.465300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1283/2537] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:00 2022] DELETE /identity/v3/projects/97f264b2bb9b4bdc92e93e07a7ad4d2d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:00:00.470844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e52de4-b34c-447a-881d-9a19027729d8 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.471525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e52de4-b34c-447a-881d-9a19027729d8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.472079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e52de4-b34c-447a-881d-9a19027729d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.496512 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f3e52de4-b34c-447a-881d-9a19027729d8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:00.497660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f3e52de4-b34c-447a-881d-9a19027729d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:00.560617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1255/2538] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:00.569250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3341001-8d53-4869-a65a-956ef09f13c0 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.575585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3341001-8d53-4869-a65a-956ef09f13c0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.575585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3341001-8d53-4869-a65a-956ef09f13c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.580176 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e52de4-b34c-447a-881d-9a19027729d8 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:00.581907 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f3e52de4-b34c-447a-881d-9a19027729d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:00.591342 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-b3341001-8d53-4869-a65a-956ef09f13c0 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:00.592461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b3341001-8d53-4869-a65a-956ef09f13c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:00.596516 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e52de4-b34c-447a-881d-9a19027729d8 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:00.601249 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:00.601888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.602164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.602437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] PATH_INFO: `/v3/projects/4298378c160f4e4785d8794e82217dfe` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.605786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1284/2539] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:00.605910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4298378c160f4e4785d8794e82217dfe)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:00.605910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:00.610585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3341001-8d53-4869-a65a-956ef09f13c0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:00.617477 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3341001-8d53-4869-a65a-956ef09f13c0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:00.622396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f49280-fd77-4cef-b0d1-57a040f86644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:00.623035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f49280-fd77-4cef-b0d1-57a040f86644 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.623319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f49280-fd77-4cef-b0d1-57a040f86644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.623593 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f49280-fd77-4cef-b0d1-57a040f86644 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.624272 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f49280-fd77-4cef-b0d1-57a040f86644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:00:00.624579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f49280-fd77-4cef-b0d1-57a040f86644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:00.626320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f49280-fd77-4cef-b0d1-57a040f86644 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:00.638458 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1285/2540] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:00:00.642920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f49280-fd77-4cef-b0d1-57a040f86644 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:00.652638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f49280-fd77-4cef-b0d1-57a040f86644 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:00.657912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25dab8-fc20-46ed-ae17-00cf30f1c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:00.658500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25dab8-fc20-46ed-ae17-00cf30f1c3bc admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.658815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25dab8-fc20-46ed-ae17-00cf30f1c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.659092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25dab8-fc20-46ed-ae17-00cf30f1c3bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.659784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25dab8-fc20-46ed-ae17-00cf30f1c3bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:00.661727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25dab8-fc20-46ed-ae17-00cf30f1c3bc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:00.685026 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:00.703035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4298378c160f4e4785d8794e82217dfe', 'request_id': 'req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.769785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4298378c160f4e4785d8794e82217dfe', 'request_id': 'req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.777027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4298378c160f4e4785d8794e82217dfe', 'request_id': 'req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.778242 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1286/2541] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:00 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:00.788355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] The token cache is being invalidate because project 4298378c160f4e4785d8794e82217dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:00.788647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.789423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4dc0a47c-e8c6-455e-84e8-083d3f4419bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:00.790333 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25dab8-fc20-46ed-ae17-00cf30f1c3bc admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:00.791258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1256/2542] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:00 2022] DELETE /identity/v3/projects/4298378c160f4e4785d8794e82217dfe => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:00.792331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7f25dab8-fc20-46ed-ae17-00cf30f1c3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:00.795603 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2986572-1999-4100-b5e2-4ab058be7776 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.796014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2986572-1999-4100-b5e2-4ab058be7776 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.796306 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2986572-1999-4100-b5e2-4ab058be7776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.817201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f2986572-1999-4100-b5e2-4ab058be7776 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:00.818363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f2986572-1999-4100-b5e2-4ab058be7776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:00.860232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25dab8-fc20-46ed-ae17-00cf30f1c3bc admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:00.865440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5d92-263d-4a1b-9c3e-327050772630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:00.866131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5d92-263d-4a1b-9c3e-327050772630 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.866430 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5d92-263d-4a1b-9c3e-327050772630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.866723 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5d92-263d-4a1b-9c3e-327050772630 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.867365 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc5d92-263d-4a1b-9c3e-327050772630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:00.869324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc5d92-263d-4a1b-9c3e-327050772630 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:00.882328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1257/2543] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:00:00.890594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df8ff1-806d-4830-8c3b-18a166b205d6 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.890969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df8ff1-806d-4830-8c3b-18a166b205d6 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.891279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df8ff1-806d-4830-8c3b-18a166b205d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.911753 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-44df8ff1-806d-4830-8c3b-18a166b205d6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:00.912963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-44df8ff1-806d-4830-8c3b-18a166b205d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:00.920898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1258/2544] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:00:00.925684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df8ff1-806d-4830-8c3b-18a166b205d6 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:00.934858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df8ff1-806d-4830-8c3b-18a166b205d6 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:00.940426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ecf73-5427-4a19-ae12-460335bdc6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:00.941032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ecf73-5427-4a19-ae12-460335bdc6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.941315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ecf73-5427-4a19-ae12-460335bdc6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.941594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ecf73-5427-4a19-ae12-460335bdc6b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.942247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ecf73-5427-4a19-ae12-460335bdc6b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:00:00.942595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ecf73-5427-4a19-ae12-460335bdc6b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:00.944211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ecf73-5427-4a19-ae12-460335bdc6b6 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:00.956301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1259/2545] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:00:00.960585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ecf73-5427-4a19-ae12-460335bdc6b6 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:00.971508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ecf73-5427-4a19-ae12-460335bdc6b6 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:00.976721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cae2d-66fc-46a2-9307-e495b5ad522f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:00.977737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cae2d-66fc-46a2-9307-e495b5ad522f admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:00.978025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cae2d-66fc-46a2-9307-e495b5ad522f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:00.978303 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cae2d-66fc-46a2-9307-e495b5ad522f admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:00.978968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cae2d-66fc-46a2-9307-e495b5ad522f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:00.981043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cae2d-66fc-46a2-9307-e495b5ad522f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:00.981664 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1287/2546] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:00 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:00.986194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5d92-263d-4a1b-9c3e-327050772630 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:00.997341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5d92-263d-4a1b-9c3e-327050772630 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.010578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92da44-79a7-4262-bbad-8ffe81be15b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.011209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92da44-79a7-4262-bbad-8ffe81be15b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.011488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92da44-79a7-4262-bbad-8ffe81be15b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.011761 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92da44-79a7-4262-bbad-8ffe81be15b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.012480 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92da44-79a7-4262-bbad-8ffe81be15b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.014155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92da44-79a7-4262-bbad-8ffe81be15b1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:01.025512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1288/2547] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:01.029603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92da44-79a7-4262-bbad-8ffe81be15b1 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:01.040222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92da44-79a7-4262-bbad-8ffe81be15b1 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.045058 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca386e-8bab-40df-b22c-c206926d0935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.045652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca386e-8bab-40df-b22c-c206926d0935 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.045939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca386e-8bab-40df-b22c-c206926d0935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.046217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca386e-8bab-40df-b22c-c206926d0935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.046964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca386e-8bab-40df-b22c-c206926d0935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.049145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca386e-8bab-40df-b22c-c206926d0935 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:01.060573 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1289/2548] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:00:01.065442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca386e-8bab-40df-b22c-c206926d0935 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:01.075114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca386e-8bab-40df-b22c-c206926d0935 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.082393 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d642d68-047f-4db0-a0f4-02ef4117896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.083081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d642d68-047f-4db0-a0f4-02ef4117896b admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.083369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d642d68-047f-4db0-a0f4-02ef4117896b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.083686 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d642d68-047f-4db0-a0f4-02ef4117896b admin admin] PATH_INFO: `/v3/projects/3df0608237f4438eaacf9dffeb0b8f46/users/6c6edc44572c4d49bf447b177c8e910f/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.086185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1260/2549] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:00 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:01.090365 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cae2d-66fc-46a2-9307-e495b5ad522f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:01.103631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d642d68-047f-4db0-a0f4-02ef4117896b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df0608237f4438eaacf9dffeb0b8f46, user_id=6c6edc44572c4d49bf447b177c8e910f, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.106313 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d642d68-047f-4db0-a0f4-02ef4117896b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:01.158223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cae2d-66fc-46a2-9307-e495b5ad522f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.163329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa374e3-46e6-4187-84e0-1a3b30d07933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.167535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa374e3-46e6-4187-84e0-1a3b30d07933 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.167535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa374e3-46e6-4187-84e0-1a3b30d07933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.167535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa374e3-46e6-4187-84e0-1a3b30d07933 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.167535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa374e3-46e6-4187-84e0-1a3b30d07933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.167535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa374e3-46e6-4187-84e0-1a3b30d07933 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:01.204511 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1290/2550] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:01 2022] PUT /identity/v3/projects/3df0608237f4438eaacf9dffeb0b8f46/users/6c6edc44572c4d49bf447b177c8e910f/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:00:01.214964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a691-4c9d-4a53-9aba-0b9968216a01 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.215253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a691-4c9d-4a53-9aba-0b9968216a01 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.215529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a691-4c9d-4a53-9aba-0b9968216a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.243986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-3b77a691-4c9d-4a53-9aba-0b9968216a01 None None] MFA Rules not processed for user `6c6edc44572c4d49bf447b177c8e910f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:01.245193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3b77a691-4c9d-4a53-9aba-0b9968216a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:01.296122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1261/2551] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:01 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:01.300142 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa374e3-46e6-4187-84e0-1a3b30d07933 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:01.308578 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1291/2552] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:00:01.310037 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa374e3-46e6-4187-84e0-1a3b30d07933 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.320494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77a691-4c9d-4a53-9aba-0b9968216a01 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:01.321925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3b77a691-4c9d-4a53-9aba-0b9968216a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:01.371520 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dba63c-6050-46a3-a29f-5afe46242351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.372135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dba63c-6050-46a3-a29f-5afe46242351 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.372419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dba63c-6050-46a3-a29f-5afe46242351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.372697 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dba63c-6050-46a3-a29f-5afe46242351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.373344 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dba63c-6050-46a3-a29f-5afe46242351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.374992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dba63c-6050-46a3-a29f-5afe46242351 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:01.381605 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77a691-4c9d-4a53-9aba-0b9968216a01 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.385264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab593584-a1c4-435d-8c19-eabfa9d41f75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.385505 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1262/2553] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:00:01.386176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab593584-a1c4-435d-8c19-eabfa9d41f75 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.386454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab593584-a1c4-435d-8c19-eabfa9d41f75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.386783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab593584-a1c4-435d-8c19-eabfa9d41f75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.389619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dba63c-6050-46a3-a29f-5afe46242351 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:01.396450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab593584-a1c4-435d-8c19-eabfa9d41f75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.397943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab593584-a1c4-435d-8c19-eabfa9d41f75 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:01.399898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dba63c-6050-46a3-a29f-5afe46242351 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.404769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64175feb-1bf0-43c6-9fab-c91522d4f3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.405363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64175feb-1bf0-43c6-9fab-c91522d4f3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.405649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64175feb-1bf0-43c6-9fab-c91522d4f3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.405927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64175feb-1bf0-43c6-9fab-c91522d4f3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.406638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64175feb-1bf0-43c6-9fab-c91522d4f3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.407812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1292/2554] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:00:01.408351 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64175feb-1bf0-43c6-9fab-c91522d4f3a9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:01.419156 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1263/2555] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:00:01.423557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab593584-a1c4-435d-8c19-eabfa9d41f75 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:01.433993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab593584-a1c4-435d-8c19-eabfa9d41f75 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.439150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5bd8-2268-42e5-8af8-97390de7dcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.442028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5bd8-2268-42e5-8af8-97390de7dcde admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.442028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5bd8-2268-42e5-8af8-97390de7dcde admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.442028 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5bd8-2268-42e5-8af8-97390de7dcde admin admin] PATH_INFO: `/v3/projects/515d73ab44b245f181efc54779e1a79a/users/f3a4f649cd5849ee97a7e6a955ba3a04/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.455808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe5bd8-2268-42e5-8af8-97390de7dcde admin admin] RBAC: Authorizing `identity:create_grant(project_id=515d73ab44b245f181efc54779e1a79a, user_id=f3a4f649cd5849ee97a7e6a955ba3a04, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.458383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe5bd8-2268-42e5-8af8-97390de7dcde admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:01.489716 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=5,keystone:SELECT=89,keystone:INSERT=6 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:01.529383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1293/2556] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:01 2022] PUT /identity/v3/projects/515d73ab44b245f181efc54779e1a79a/users/f3a4f649cd5849ee97a7e6a955ba3a04/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:00:01.541894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e341e5-acdb-4cfe-b262-b667d9428c5b None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.543825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e341e5-acdb-4cfe-b262-b667d9428c5b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.545271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e341e5-acdb-4cfe-b262-b667d9428c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.585279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-f8e341e5-acdb-4cfe-b262-b667d9428c5b None None] MFA Rules not processed for user `f3a4f649cd5849ee97a7e6a955ba3a04`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:01.585279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-f8e341e5-acdb-4cfe-b262-b667d9428c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:01.649442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1264/2557] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:01.654086 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5bd8-2268-42e5-8af8-97390de7dcde admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:01.690638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:01.721181 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5bd8-2268-42e5-8af8-97390de7dcde admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.725915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287ca34-68dc-439a-9983-10b8e36c0ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.726545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287ca34-68dc-439a-9983-10b8e36c0ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.726835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287ca34-68dc-439a-9983-10b8e36c0ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.727111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287ca34-68dc-439a-9983-10b8e36c0ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.727751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287ca34-68dc-439a-9983-10b8e36c0ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.729672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287ca34-68dc-439a-9983-10b8e36c0ae3 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:01.844214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1294/2558] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:01 2022] POST /identity/v3/projects => generated 379 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:01.844391 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e341e5-acdb-4cfe-b262-b667d9428c5b None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:01.854121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e341e5-acdb-4cfe-b262-b667d9428c5b None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:01.910357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1ac00-9cbd-46e9-8995-29ae93d7973f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:01.911055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1ac00-9cbd-46e9-8995-29ae93d7973f admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:01.911334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1ac00-9cbd-46e9-8995-29ae93d7973f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:01.911605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1ac00-9cbd-46e9-8995-29ae93d7973f admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:01.912240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1ac00-9cbd-46e9-8995-29ae93d7973f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:01.914357 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1ac00-9cbd-46e9-8995-29ae93d7973f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:02.022262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1265/2559] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:01 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:02.026779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287ca34-68dc-439a-9983-10b8e36c0ae3 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:02.037463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287ca34-68dc-439a-9983-10b8e36c0ae3 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:02.042088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3ba2e-2432-4197-a6f8-4bd71d5f5288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:02.042734 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3ba2e-2432-4197-a6f8-4bd71d5f5288 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:02.043017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3ba2e-2432-4197-a6f8-4bd71d5f5288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:02.043289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3ba2e-2432-4197-a6f8-4bd71d5f5288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:02.043933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3ba2e-2432-4197-a6f8-4bd71d5f5288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:02.045539 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3ba2e-2432-4197-a6f8-4bd71d5f5288 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:02.056713 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1295/2560] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:02.060944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1ac00-9cbd-46e9-8995-29ae93d7973f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:02.074259 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1ac00-9cbd-46e9-8995-29ae93d7973f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:02.080188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab708274-396f-4bae-9e5c-327cfdc34aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:02.080862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab708274-396f-4bae-9e5c-327cfdc34aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:02.081147 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab708274-396f-4bae-9e5c-327cfdc34aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:02.081449 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab708274-396f-4bae-9e5c-327cfdc34aa7 admin admin] PATH_INFO: `/v3/projects/bb64930232ed43898ba291b6a063b43d/users/3c9670d935e34ce9899a3f4dd56a315e/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:02.098540 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab708274-396f-4bae-9e5c-327cfdc34aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb64930232ed43898ba291b6a063b43d, user_id=3c9670d935e34ce9899a3f4dd56a315e, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:02.101218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab708274-396f-4bae-9e5c-327cfdc34aa7 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:02.195973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:02.201595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1266/2561] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:02 2022] PUT /identity/v3/projects/bb64930232ed43898ba291b6a063b43d/users/3c9670d935e34ce9899a3f4dd56a315e/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:02.207280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3ba2e-2432-4197-a6f8-4bd71d5f5288 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:02.217637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3ba2e-2432-4197-a6f8-4bd71d5f5288 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:02.221977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:02.276304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a224f-813b-4beb-913c-534458188d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:02.276916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a224f-813b-4beb-913c-534458188d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:02.277197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a224f-813b-4beb-913c-534458188d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:02.277476 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a224f-813b-4beb-913c-534458188d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:02.278149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a224f-813b-4beb-913c-534458188d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:02.279784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a224f-813b-4beb-913c-534458188d39 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:02.291341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1296/2562] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:02.295601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab708274-396f-4bae-9e5c-327cfdc34aa7 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:02.310726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab708274-396f-4bae-9e5c-327cfdc34aa7 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:02.311777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8b8b7-e09b-453f-a2ea-b6f265f67bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:02.312441 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8b8b7-e09b-453f-a2ea-b6f265f67bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:02.312757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8b8b7-e09b-453f-a2ea-b6f265f67bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:02.313470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8b8b7-e09b-453f-a2ea-b6f265f67bbe admin admin] PATH_INFO: `/v3/projects/bb64930232ed43898ba291b6a063b43d/users/3c9670d935e34ce9899a3f4dd56a315e/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:02.315124 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8b8b7-e09b-453f-a2ea-b6f265f67bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb64930232ed43898ba291b6a063b43d, user_id=3c9670d935e34ce9899a3f4dd56a315e, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:02.317795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8b8b7-e09b-453f-a2ea-b6f265f67bbe admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:02.343578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1267/2563] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:02 2022] PUT /identity/v3/projects/bb64930232ed43898ba291b6a063b43d/users/3c9670d935e34ce9899a3f4dd56a315e/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:00:02.357002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b150f8-f8ed-44c2-a88b-22066331bf27 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:02.357002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b150f8-f8ed-44c2-a88b-22066331bf27 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:02.357002 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b150f8-f8ed-44c2-a88b-22066331bf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:02.397571 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e4b150f8-f8ed-44c2-a88b-22066331bf27 None None] MFA Rules not processed for user `3c9670d935e34ce9899a3f4dd56a315e`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:02.399162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e4b150f8-f8ed-44c2-a88b-22066331bf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:02.496098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1297/2564] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:02.507658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8b8b7-e09b-453f-a2ea-b6f265f67bbe admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:02.565188 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8b8b7-e09b-453f-a2ea-b6f265f67bbe admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:02.568927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108621f-8b98-4953-84f9-69d9e7f1e5ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:02.569532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108621f-8b98-4953-84f9-69d9e7f1e5ee service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:02.569815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108621f-8b98-4953-84f9-69d9e7f1e5ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:02.570092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108621f-8b98-4953-84f9-69d9e7f1e5ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:02.580298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108621f-8b98-4953-84f9-69d9e7f1e5ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:02.582963 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108621f-8b98-4953-84f9-69d9e7f1e5ee service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:02.591298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1268/2565] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:03.161136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b150f8-f8ed-44c2-a88b-22066331bf27 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:03.220943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:03.245157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b150f8-f8ed-44c2-a88b-22066331bf27 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:03.251019 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:03.251736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:03.251890 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:03.252098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] PATH_INFO: `/v3/users/f3a4f649cd5849ee97a7e6a955ba3a04` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:03.253532 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a4f649cd5849ee97a7e6a955ba3a04)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:03.255507 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:03.437330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a4f649cd5849ee97a7e6a955ba3a04'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:03.439088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a4f649cd5849ee97a7e6a955ba3a04'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:03.488992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a4f649cd5849ee97a7e6a955ba3a04'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:03.496398 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a4f649cd5849ee97a7e6a955ba3a04'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:03.509478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a4f649cd5849ee97a7e6a955ba3a04'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:03.515573 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1298/2566] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:03 2022] DELETE /identity/v3/users/f3a4f649cd5849ee97a7e6a955ba3a04 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:03.520235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450fb45-f9ea-4690-82a9-2a05d2143a1a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:03.520711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450fb45-f9ea-4690-82a9-2a05d2143a1a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:03.520833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450fb45-f9ea-4690-82a9-2a05d2143a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:03.549842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-3450fb45-f9ea-4690-82a9-2a05d2143a1a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:03.551161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3450fb45-f9ea-4690-82a9-2a05d2143a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:03.612798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1269/2567] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:00:03.622657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:03.624072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:03.697976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda94fd-546b-4241-888a-6ee511f3085b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:03.701828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b9e29-075b-4b9a-97a2-c93e41f05712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:03.702696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b9e29-075b-4b9a-97a2-c93e41f05712 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:03.702982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b9e29-075b-4b9a-97a2-c93e41f05712 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:03.703355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b9e29-075b-4b9a-97a2-c93e41f05712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:03.712599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b9e29-075b-4b9a-97a2-c93e41f05712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:03.714067 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b9e29-075b-4b9a-97a2-c93e41f05712 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:03.724145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1299/2568] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:03.845210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705158d6-433b-4829-b6a3-106911113c31 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:03.845587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705158d6-433b-4829-b6a3-106911113c31 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:03.845873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705158d6-433b-4829-b6a3-106911113c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:03.867308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-705158d6-433b-4829-b6a3-106911113c31 None None] MFA Rules not processed for user `2206cb5005354ab4a716efb525c633e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:03.868572 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-705158d6-433b-4829-b6a3-106911113c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:03.881206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:03.925871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1270/2569] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:04.205378 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b9e29-075b-4b9a-97a2-c93e41f05712 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:04.207536 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-485b9e29-075b-4b9a-97a2-c93e41f05712 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:04.223848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b9e29-075b-4b9a-97a2-c93e41f05712 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:04.229033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:04.229671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:04.230127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:04.230590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] PATH_INFO: `/v3/projects/515d73ab44b245f181efc54779e1a79a` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:04.231911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=515d73ab44b245f181efc54779e1a79a)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:04.233870 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:04.345294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:04.358721 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '515d73ab44b245f181efc54779e1a79a', 'request_id': 'req-467cc622-7cd0-4b37-96a8-3badbe76f8cc'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.367144 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '515d73ab44b245f181efc54779e1a79a', 'request_id': 'req-467cc622-7cd0-4b37-96a8-3badbe76f8cc'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.481780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '515d73ab44b245f181efc54779e1a79a', 'request_id': 'req-467cc622-7cd0-4b37-96a8-3badbe76f8cc'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.491362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] The token cache is being invalidate because project 515d73ab44b245f181efc54779e1a79a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:04.491706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.492362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.493879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1300/2570] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:04 2022] DELETE /identity/v3/projects/515d73ab44b245f181efc54779e1a79a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:04.497969 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705158d6-433b-4829-b6a3-106911113c31 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:04.509298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705158d6-433b-4829-b6a3-106911113c31 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:04.567324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:04.568125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:04.568237 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:04.568415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] PATH_INFO: `/v3/users/3c9670d935e34ce9899a3f4dd56a315e` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:04.569559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c9670d935e34ce9899a3f4dd56a315e)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:04.571543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:04.736566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c9670d935e34ce9899a3f4dd56a315e'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.798301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c9670d935e34ce9899a3f4dd56a315e'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.812112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c9670d935e34ce9899a3f4dd56a315e'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.819538 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c9670d935e34ce9899a3f4dd56a315e'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.820407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-7a985e5f-6f1f-4b4f-9ccd-24e32d6b49d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c9670d935e34ce9899a3f4dd56a315e'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:04.826853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1271/2571] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:04 2022] DELETE /identity/v3/users/3c9670d935e34ce9899a3f4dd56a315e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:05.304846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:05.306864 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:05.321915 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cc622-7cd0-4b37-96a8-3badbe76f8cc admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:05.387258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:05.387906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:05.388193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:05.388690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] PATH_INFO: `/v3/projects/bb64930232ed43898ba291b6a063b43d` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:05.389835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb64930232ed43898ba291b6a063b43d)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:05.391794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:05.558127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb64930232ed43898ba291b6a063b43d', 'request_id': 'req-7d61981f-4b1f-4524-8763-efd5a51948a4'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:05.567091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb64930232ed43898ba291b6a063b43d', 'request_id': 'req-7d61981f-4b1f-4524-8763-efd5a51948a4'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:05.655510 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb64930232ed43898ba291b6a063b43d', 'request_id': 'req-7d61981f-4b1f-4524-8763-efd5a51948a4'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:05.671940 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] The token cache is being invalidate because project bb64930232ed43898ba291b6a063b43d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:05.672430 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:05.673271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:05.674839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1301/2572] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:05 2022] DELETE /identity/v3/projects/bb64930232ed43898ba291b6a063b43d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:05.679927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605536f-14cd-414f-b066-7cdebe82177e None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:05.680409 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605536f-14cd-414f-b066-7cdebe82177e None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:05.680513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605536f-14cd-414f-b066-7cdebe82177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:05.738535 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-b605536f-14cd-414f-b066-7cdebe82177e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:05.739821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b605536f-14cd-414f-b066-7cdebe82177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:05.753969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:05.754760 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:05.755540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:05.841195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:05.842666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1272/2573] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:05.846277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61981f-4b1f-4524-8763-efd5a51948a4 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:05.848549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9dbfe-0279-418a-9b33-1d40bb97e5b1 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:05.848858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9dbfe-0279-418a-9b33-1d40bb97e5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:05.849141 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9dbfe-0279-418a-9b33-1d40bb97e5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:05.859636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:05.860881 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:05.861176 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:05.861452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] PATH_INFO: `/v3/users/6c6edc44572c4d49bf447b177c8e910f` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:05.863262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c6edc44572c4d49bf447b177c8e910f)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:05.865728 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:05.878785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2bc9dbfe-0279-418a-9b33-1d40bb97e5b1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:05.880211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2bc9dbfe-0279-418a-9b33-1d40bb97e5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:05.892433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1273/2574] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:05.899401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9dbfe-0279-418a-9b33-1d40bb97e5b1 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:05.912918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9dbfe-0279-418a-9b33-1d40bb97e5b1 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:05.917887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd041e-d38b-4ea1-bcbe-881e5fd73d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:05.918523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd041e-d38b-4ea1-bcbe-881e5fd73d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:05.918854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd041e-d38b-4ea1-bcbe-881e5fd73d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:05.919134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd041e-d38b-4ea1-bcbe-881e5fd73d5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:05.919785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd041e-d38b-4ea1-bcbe-881e5fd73d5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:00:05.920310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd041e-d38b-4ea1-bcbe-881e5fd73d5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:05.922093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd041e-d38b-4ea1-bcbe-881e5fd73d5d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:05.939206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1274/2575] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:00:05.948498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd041e-d38b-4ea1-bcbe-881e5fd73d5d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:05.960174 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd041e-d38b-4ea1-bcbe-881e5fd73d5d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:05.965242 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a849e0-cd4f-4dca-b897-36696fb3e996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:05.965880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a849e0-cd4f-4dca-b897-36696fb3e996 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:05.966192 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a849e0-cd4f-4dca-b897-36696fb3e996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:05.966444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a849e0-cd4f-4dca-b897-36696fb3e996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:05.967123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a849e0-cd4f-4dca-b897-36696fb3e996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:05.969674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a849e0-cd4f-4dca-b897-36696fb3e996 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:06.073271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1275/2576] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:05 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:06.077428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a849e0-cd4f-4dca-b897-36696fb3e996 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:06.140500 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c6edc44572c4d49bf447b177c8e910f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:06.142432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c6edc44572c4d49bf447b177c8e910f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:06.146651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a849e0-cd4f-4dca-b897-36696fb3e996 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:06.151379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb53b37-2324-4b6b-bfd2-a2d208ef904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:06.152002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb53b37-2324-4b6b-bfd2-a2d208ef904e admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:06.152287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb53b37-2324-4b6b-bfd2-a2d208ef904e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:06.152563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb53b37-2324-4b6b-bfd2-a2d208ef904e admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:06.153216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb53b37-2324-4b6b-bfd2-a2d208ef904e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:06.155198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb53b37-2324-4b6b-bfd2-a2d208ef904e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:06.272163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c6edc44572c4d49bf447b177c8e910f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:06.278009 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1276/2577] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:06 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:06.281644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c6edc44572c4d49bf447b177c8e910f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:06.282309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb53b37-2324-4b6b-bfd2-a2d208ef904e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:06.284072 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-7bb53b37-2324-4b6b-bfd2-a2d208ef904e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:06.294085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5f145a11-e798-4082-bd6d-d9d8e20a5eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c6edc44572c4d49bf447b177c8e910f'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:06.297329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb53b37-2324-4b6b-bfd2-a2d208ef904e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:06.303947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973b9a4-83c2-43e0-8cc6-7947cf40d4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:06.304593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973b9a4-83c2-43e0-8cc6-7947cf40d4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:06.304939 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973b9a4-83c2-43e0-8cc6-7947cf40d4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:06.305283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973b9a4-83c2-43e0-8cc6-7947cf40d4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:06.311508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973b9a4-83c2-43e0-8cc6-7947cf40d4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:06.311636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1302/2578] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:05 2022] DELETE /identity/v3/users/6c6edc44572c4d49bf447b177c8e910f => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:06.312111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973b9a4-83c2-43e0-8cc6-7947cf40d4fd admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:06.314085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c95984-c11d-4d55-a83a-03076234eb66 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:06.314085 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c95984-c11d-4d55-a83a-03076234eb66 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:06.314302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c95984-c11d-4d55-a83a-03076234eb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:06.331079 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1277/2579] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:00:06.332972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973b9a4-83c2-43e0-8cc6-7947cf40d4fd admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:06.343872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-63c95984-c11d-4d55-a83a-03076234eb66 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:06.345453 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-63c95984-c11d-4d55-a83a-03076234eb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:06.349870 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973b9a4-83c2-43e0-8cc6-7947cf40d4fd admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:06.418974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da9c8d-b48b-4f97-a76a-2a04ba7ad83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:06.420207 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da9c8d-b48b-4f97-a76a-2a04ba7ad83d admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:06.420493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da9c8d-b48b-4f97-a76a-2a04ba7ad83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:06.420772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da9c8d-b48b-4f97-a76a-2a04ba7ad83d admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:06.421461 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da9c8d-b48b-4f97-a76a-2a04ba7ad83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:06.424663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da9c8d-b48b-4f97-a76a-2a04ba7ad83d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:06.427966 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1303/2580] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:00:06.438539 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1278/2581] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:06.439926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c95984-c11d-4d55-a83a-03076234eb66 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:06.441239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da9c8d-b48b-4f97-a76a-2a04ba7ad83d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:06.442791 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-63c95984-c11d-4d55-a83a-03076234eb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:06.451643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da9c8d-b48b-4f97-a76a-2a04ba7ad83d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:06.459464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ef7fe-9fd4-4d94-a231-9683d5a5ee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:06.460307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ef7fe-9fd4-4d94-a231-9683d5a5ee85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:06.460775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ef7fe-9fd4-4d94-a231-9683d5a5ee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:06.461151 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ef7fe-9fd4-4d94-a231-9683d5a5ee85 admin admin] PATH_INFO: `/v3/projects/8146fb912c2242b88808f85485ef32ed/users/ef7c640e8f5a425683c55594646f6e00/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:06.477240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ef7fe-9fd4-4d94-a231-9683d5a5ee85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8146fb912c2242b88808f85485ef32ed, user_id=ef7c640e8f5a425683c55594646f6e00, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:06.480138 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ef7fe-9fd4-4d94-a231-9683d5a5ee85 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:06.512591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c95984-c11d-4d55-a83a-03076234eb66 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:06.516726 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff3701-2448-4c40-9152-e0b207c22771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:06.517485 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff3701-2448-4c40-9152-e0b207c22771 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:06.517772 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff3701-2448-4c40-9152-e0b207c22771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:06.518045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff3701-2448-4c40-9152-e0b207c22771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:06.527259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff3701-2448-4c40-9152-e0b207c22771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:06.528762 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff3701-2448-4c40-9152-e0b207c22771 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:06.538591 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1304/2582] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:06.595027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1279/2583] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:06 2022] PUT /identity/v3/projects/8146fb912c2242b88808f85485ef32ed/users/ef7c640e8f5a425683c55594646f6e00/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:06.605584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7393cc-18a9-41a5-ad03-37e6c203c513 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:06.605880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7393cc-18a9-41a5-ad03-37e6c203c513 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:06.606159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7393cc-18a9-41a5-ad03-37e6c203c513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:06.636624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9a7393cc-18a9-41a5-ad03-37e6c203c513 None None] MFA Rules not processed for user `ef7c640e8f5a425683c55594646f6e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:06.638134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9a7393cc-18a9-41a5-ad03-37e6c203c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:06.704043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1305/2584] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:06.708679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ef7fe-9fd4-4d94-a231-9683d5a5ee85 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:06.790435 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ef7fe-9fd4-4d94-a231-9683d5a5ee85 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:06.796271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bde81-b6cc-4e91-bc4e-88b95263cefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:06.797483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bde81-b6cc-4e91-bc4e-88b95263cefa admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:06.798011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bde81-b6cc-4e91-bc4e-88b95263cefa admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:06.798349 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bde81-b6cc-4e91-bc4e-88b95263cefa admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:06.798994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858bde81-b6cc-4e91-bc4e-88b95263cefa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:06.801025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858bde81-b6cc-4e91-bc4e-88b95263cefa admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:06.825190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:06.914289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1280/2585] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:06 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:06.915474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7393cc-18a9-41a5-ad03-37e6c203c513 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:06.928863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7393cc-18a9-41a5-ad03-37e6c203c513 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:06.943972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:06.993076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a0adf-b16c-402e-80da-984631b8c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:06.993710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a0adf-b16c-402e-80da-984631b8c9ac admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:06.993998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a0adf-b16c-402e-80da-984631b8c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:06.994304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a0adf-b16c-402e-80da-984631b8c9ac admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:06.994965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a0adf-b16c-402e-80da-984631b8c9ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.001445 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a0adf-b16c-402e-80da-984631b8c9ac admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:07.033110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bde81-b6cc-4e91-bc4e-88b95263cefa admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:07.046442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-858bde81-b6cc-4e91-bc4e-88b95263cefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:07.076654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bde81-b6cc-4e91-bc4e-88b95263cefa admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:07.094307 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:07.095507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.095986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.096509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] PATH_INFO: `/v3/projects/3df0608237f4438eaacf9dffeb0b8f46` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.097874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df0608237f4438eaacf9dffeb0b8f46)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.101425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:07.110384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1306/2586] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:06 2022] POST /identity/v3/users => generated 296 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:07.114331 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a0adf-b16c-402e-80da-984631b8c9ac admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:07.129298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a0adf-b16c-402e-80da-984631b8c9ac admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:07.136385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb2cbe-48d4-41b7-9997-6d6e1abf2b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:07.137470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb2cbe-48d4-41b7-9997-6d6e1abf2b26 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.137946 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb2cbe-48d4-41b7-9997-6d6e1abf2b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.138410 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb2cbe-48d4-41b7-9997-6d6e1abf2b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.139337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb2cbe-48d4-41b7-9997-6d6e1abf2b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.142471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb2cbe-48d4-41b7-9997-6d6e1abf2b26 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:07.155783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1307/2587] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:07.161152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb2cbe-48d4-41b7-9997-6d6e1abf2b26 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:07.177307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb2cbe-48d4-41b7-9997-6d6e1abf2b26 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:07.189036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4409c8-f905-4086-9e98-87ea5e3659ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:07.189878 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4409c8-f905-4086-9e98-87ea5e3659ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.190336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4409c8-f905-4086-9e98-87ea5e3659ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.190652 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4409c8-f905-4086-9e98-87ea5e3659ed admin admin] PATH_INFO: `/v3/projects/7ff47887cb9841cca42c4f7caf8490cb/users/f4ca8dce5b3a4bfdb2648a44e0cac17f/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.214183 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4409c8-f905-4086-9e98-87ea5e3659ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff47887cb9841cca42c4f7caf8490cb, user_id=f4ca8dce5b3a4bfdb2648a44e0cac17f, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.216974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4409c8-f905-4086-9e98-87ea5e3659ed admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:07.325660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1308/2588] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:07 2022] PUT /identity/v3/projects/7ff47887cb9841cca42c4f7caf8490cb/users/f4ca8dce5b3a4bfdb2648a44e0cac17f/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:07.331390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4409c8-f905-4086-9e98-87ea5e3659ed admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:07.340175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df0608237f4438eaacf9dffeb0b8f46', 'request_id': 'req-420e2f76-b56f-4648-8870-88249ff936de'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:07.400995 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4409c8-f905-4086-9e98-87ea5e3659ed admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:07.437016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df0608237f4438eaacf9dffeb0b8f46', 'request_id': 'req-420e2f76-b56f-4648-8870-88249ff936de'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:07.446058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df0608237f4438eaacf9dffeb0b8f46', 'request_id': 'req-420e2f76-b56f-4648-8870-88249ff936de'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:07.457721 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] The token cache is being invalidate because project 3df0608237f4438eaacf9dffeb0b8f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:07.458019 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:07.458710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-420e2f76-b56f-4648-8870-88249ff936de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:07.460611 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1281/2589] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:07 2022] DELETE /identity/v3/projects/3df0608237f4438eaacf9dffeb0b8f46 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:07.466042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cf704-01bf-482c-829a-ec63c546ebb8 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.466042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cf704-01bf-482c-829a-ec63c546ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.466318 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cf704-01bf-482c-829a-ec63c546ebb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.470985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9a24e-2a98-4a5a-a729-f6c5880c6f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:07.472564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9a24e-2a98-4a5a-a729-f6c5880c6f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.472875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9a24e-2a98-4a5a-a729-f6c5880c6f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.475107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9a24e-2a98-4a5a-a729-f6c5880c6f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.475107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9a24e-2a98-4a5a-a729-f6c5880c6f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.476707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9a24e-2a98-4a5a-a729-f6c5880c6f08 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:07.488201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-321cf704-01bf-482c-829a-ec63c546ebb8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:07.488654 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1309/2590] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:07.489775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-321cf704-01bf-482c-829a-ec63c546ebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:07.492805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9a24e-2a98-4a5a-a729-f6c5880c6f08 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:07.505316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3bd9a24e-2a98-4a5a-a729-f6c5880c6f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:07.516611 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1282/2591] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:07.518133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba00692-b909-456d-aba8-0ad7fbba3180 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.518492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba00692-b909-456d-aba8-0ad7fbba3180 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.518880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba00692-b909-456d-aba8-0ad7fbba3180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.521206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9a24e-2a98-4a5a-a729-f6c5880c6f08 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:07.526938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47508b83-f0d9-4fbb-8741-3b8efbeceb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:07.527529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47508b83-f0d9-4fbb-8741-3b8efbeceb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.527712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47508b83-f0d9-4fbb-8741-3b8efbeceb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.527990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47508b83-f0d9-4fbb-8741-3b8efbeceb00 admin admin] PATH_INFO: `/v3/projects/7ff47887cb9841cca42c4f7caf8490cb/users/f4ca8dce5b3a4bfdb2648a44e0cac17f/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.529964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47508b83-f0d9-4fbb-8741-3b8efbeceb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff47887cb9841cca42c4f7caf8490cb, user_id=f4ca8dce5b3a4bfdb2648a44e0cac17f, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.533339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47508b83-f0d9-4fbb-8741-3b8efbeceb00 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:07.548169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-dba00692-b909-456d-aba8-0ad7fbba3180 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:07.550891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dba00692-b909-456d-aba8-0ad7fbba3180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:07.553635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1310/2592] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:07 2022] PUT /identity/v3/projects/7ff47887cb9841cca42c4f7caf8490cb/users/f4ca8dce5b3a4bfdb2648a44e0cac17f/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:07.555280 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e65ab-d821-4c0a-a806-8828b2c76f6c None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.555584 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e65ab-d821-4c0a-a806-8828b2c76f6c None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.555889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e65ab-d821-4c0a-a806-8828b2c76f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.586902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-533e65ab-d821-4c0a-a806-8828b2c76f6c None None] MFA Rules not processed for user `f4ca8dce5b3a4bfdb2648a44e0cac17f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:07.588137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-533e65ab-d821-4c0a-a806-8828b2c76f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:07.590288 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:07.639981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1283/2593] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:07.648606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba00692-b909-456d-aba8-0ad7fbba3180 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:07.657112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba00692-b909-456d-aba8-0ad7fbba3180 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:07.662341 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:07.663000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.663286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.663566 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.664723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:00:07.664723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.666247 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:07.677139 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1311/2594] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:07.679290 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1284/2595] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:00:07.683423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e65ab-d821-4c0a-a806-8828b2c76f6c None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:07.688743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:07.697797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e65ab-d821-4c0a-a806-8828b2c76f6c None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:07.702261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:07.704159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2e0c0-25e1-4be4-9e9a-c7a9247c4e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:07.704610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2e0c0-25e1-4be4-9e9a-c7a9247c4e27 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.704872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2e0c0-25e1-4be4-9e9a-c7a9247c4e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.705138 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2e0c0-25e1-4be4-9e9a-c7a9247c4e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.705818 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2e0c0-25e1-4be4-9e9a-c7a9247c4e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.707879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2e0c0-25e1-4be4-9e9a-c7a9247c4e27 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:07.768615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60e56a-e934-4bd8-812d-af2719563394 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:07.772319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce032b03-695d-49ea-b2b0-5a76a5df558c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:07.772932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce032b03-695d-49ea-b2b0-5a76a5df558c service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.773214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce032b03-695d-49ea-b2b0-5a76a5df558c service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.773490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce032b03-695d-49ea-b2b0-5a76a5df558c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.775272 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ce032b03-695d-49ea-b2b0-5a76a5df558c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:07.823247 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1312/2596] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:07 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:07.826367 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2e0c0-25e1-4be4-9e9a-c7a9247c4e27 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:07.841177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce032b03-695d-49ea-b2b0-5a76a5df558c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.842952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce032b03-695d-49ea-b2b0-5a76a5df558c service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:07.853244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1285/2597] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:00:07.892449 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2e0c0-25e1-4be4-9e9a-c7a9247c4e27 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:07.897269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77931e-fce3-4995-bc8f-496899ec82fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:07.897880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77931e-fce3-4995-bc8f-496899ec82fa admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:07.898164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77931e-fce3-4995-bc8f-496899ec82fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:07.898441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77931e-fce3-4995-bc8f-496899ec82fa admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:07.899111 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77931e-fce3-4995-bc8f-496899ec82fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:07.901177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77931e-fce3-4995-bc8f-496899ec82fa admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:08.035623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:08.041444 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1313/2598] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:07 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:08.046205 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce032b03-695d-49ea-b2b0-5a76a5df558c service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:08.059904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce032b03-695d-49ea-b2b0-5a76a5df558c service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:08.065862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b4052-b27c-49b9-a01d-2e379809799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:08.066569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b4052-b27c-49b9-a01d-2e379809799b admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:08.066953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b4052-b27c-49b9-a01d-2e379809799b admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:08.067334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b4052-b27c-49b9-a01d-2e379809799b admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:08.068883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b4052-b27c-49b9-a01d-2e379809799b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:08.069933 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b4052-b27c-49b9-a01d-2e379809799b admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:08.082271 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1286/2599] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:08.085843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77931e-fce3-4995-bc8f-496899ec82fa admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:08.100827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77931e-fce3-4995-bc8f-496899ec82fa admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:08.106540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437afd8-47d3-4f12-bbfc-25bb169a8747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:08.107164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437afd8-47d3-4f12-bbfc-25bb169a8747 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:08.107452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437afd8-47d3-4f12-bbfc-25bb169a8747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:08.107731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437afd8-47d3-4f12-bbfc-25bb169a8747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:08.108416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437afd8-47d3-4f12-bbfc-25bb169a8747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:08.110912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437afd8-47d3-4f12-bbfc-25bb169a8747 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:08.132278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1314/2600] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:08.132431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b4052-b27c-49b9-a01d-2e379809799b admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:08.140920 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b4052-b27c-49b9-a01d-2e379809799b admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:08.145890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9976954-c427-43ba-9836-95c5f450a47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:08.146595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9976954-c427-43ba-9836-95c5f450a47b admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:08.147047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9976954-c427-43ba-9836-95c5f450a47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:08.147332 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9976954-c427-43ba-9836-95c5f450a47b admin admin] PATH_INFO: `/v3/projects/cb1a69cce5cd4a8393d04fdcbe9dea6f/users/59ce5bae5ff1444883b78f78abda74ea/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:08.167573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9976954-c427-43ba-9836-95c5f450a47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1a69cce5cd4a8393d04fdcbe9dea6f, user_id=59ce5bae5ff1444883b78f78abda74ea, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:08.170418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9976954-c427-43ba-9836-95c5f450a47b admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:08.278244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1287/2601] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:08 2022] PUT /identity/v3/projects/cb1a69cce5cd4a8393d04fdcbe9dea6f/users/59ce5bae5ff1444883b78f78abda74ea/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:08.289405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57d0b8-0d9e-4ca7-b5ef-0f42740cfdd9 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:08.289811 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57d0b8-0d9e-4ca7-b5ef-0f42740cfdd9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:08.290222 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57d0b8-0d9e-4ca7-b5ef-0f42740cfdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:08.320558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5c57d0b8-0d9e-4ca7-b5ef-0f42740cfdd9 None None] MFA Rules not processed for user `59ce5bae5ff1444883b78f78abda74ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:08.321795 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5c57d0b8-0d9e-4ca7-b5ef-0f42740cfdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:08.390796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1315/2602] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:08.403803 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9976954-c427-43ba-9836-95c5f450a47b admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:08.406610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c9976954-c427-43ba-9836-95c5f450a47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:08.476477 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9976954-c427-43ba-9836-95c5f450a47b admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:08.482269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318835ca-4faf-4108-99ff-8e44d6b62f3e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ff3f53ddbb04eb8916b9ccfdb03dd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:08.482918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318835ca-4faf-4108-99ff-8e44d6b62f3e service glance] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:08.483200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318835ca-4faf-4108-99ff-8e44d6b62f3e service glance] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:08.483475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318835ca-4faf-4108-99ff-8e44d6b62f3e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:08.497534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318835ca-4faf-4108-99ff-8e44d6b62f3e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:08.499122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318835ca-4faf-4108-99ff-8e44d6b62f3e service glance] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:08.513492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1288/2603] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:00:08.855428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:09.038326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:13.645403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57d0b8-0d9e-4ca7-b5ef-0f42740cfdd9 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:13.723922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57d0b8-0d9e-4ca7-b5ef-0f42740cfdd9 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:13.728225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb377d9-d0d6-4254-8a15-36930e958ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:13.729036 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb377d9-d0d6-4254-8a15-36930e958ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:13.729321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb377d9-d0d6-4254-8a15-36930e958ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:13.729595 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb377d9-d0d6-4254-8a15-36930e958ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:13.803417 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb377d9-d0d6-4254-8a15-36930e958ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:13.804798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb377d9-d0d6-4254-8a15-36930e958ac5 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:13.816164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1316/2604] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:19.911899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=10 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:20.139331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318835ca-4faf-4108-99ff-8e44d6b62f3e service glance] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:20.214317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318835ca-4faf-4108-99ff-8e44d6b62f3e service glance] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:20.219208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:20.219844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:20.220126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:20.220403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] PATH_INFO: `/v3/users/ef7c640e8f5a425683c55594646f6e00` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:20.221531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7c640e8f5a425683c55594646f6e00)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:20.224309 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:20.423217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7c640e8f5a425683c55594646f6e00'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:20.482709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7c640e8f5a425683c55594646f6e00'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:20.495910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7c640e8f5a425683c55594646f6e00'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:20.503851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7c640e8f5a425683c55594646f6e00'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:20.504670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7c640e8f5a425683c55594646f6e00'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:20.511665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1289/2605] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:20 2022] DELETE /identity/v3/users/ef7c640e8f5a425683c55594646f6e00 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:20.515484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aed6e-5cea-4cbd-9a65-4d23126887ee None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:20.516072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aed6e-5cea-4cbd-9a65-4d23126887ee None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:20.516072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aed6e-5cea-4cbd-9a65-4d23126887ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:20.535195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e14aed6e-5cea-4cbd-9a65-4d23126887ee None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:20.536300 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e14aed6e-5cea-4cbd-9a65-4d23126887ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:20.545497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:20.609093 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1317/2606] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:20.620036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:20.621472 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:20.682630 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d96bc2-b6b9-43fb-b5b4-e360a7d86f79 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:20.686313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea67d9-3d00-459e-9bfb-1d56d86d91f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:20.686970 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea67d9-3d00-459e-9bfb-1d56d86d91f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:20.687292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea67d9-3d00-459e-9bfb-1d56d86d91f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:20.687570 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea67d9-3d00-459e-9bfb-1d56d86d91f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:20.697331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea67d9-3d00-459e-9bfb-1d56d86d91f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:20.703402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea67d9-3d00-459e-9bfb-1d56d86d91f2 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:20.709301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1290/2607] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:21.161178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aed6e-5cea-4cbd-9a65-4d23126887ee None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:21.163800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e14aed6e-5cea-4cbd-9a65-4d23126887ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:21.199227 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aed6e-5cea-4cbd-9a65-4d23126887ee None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:21.204104 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:21.204755 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:21.205039 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:21.205320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] PATH_INFO: `/v3/projects/8146fb912c2242b88808f85485ef32ed` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:21.207029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8146fb912c2242b88808f85485ef32ed)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:21.209501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:21.342932 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8146fb912c2242b88808f85485ef32ed', 'request_id': 'req-c1677ea1-fde1-4811-adc3-cc4f91d74cba'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.349877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8146fb912c2242b88808f85485ef32ed', 'request_id': 'req-c1677ea1-fde1-4811-adc3-cc4f91d74cba'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.408452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8146fb912c2242b88808f85485ef32ed', 'request_id': 'req-c1677ea1-fde1-4811-adc3-cc4f91d74cba'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.421939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] The token cache is being invalidate because project 8146fb912c2242b88808f85485ef32ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:21.422232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.422961 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.424437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1318/2608] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:21 2022] DELETE /identity/v3/projects/8146fb912c2242b88808f85485ef32ed => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:21.429038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea67d9-3d00-459e-9bfb-1d56d86d91f2 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:21.438854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea67d9-3d00-459e-9bfb-1d56d86d91f2 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:21.457912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:21.496500 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:21.497154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:21.497436 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:21.497709 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] PATH_INFO: `/v3/users/f4ca8dce5b3a4bfdb2648a44e0cac17f` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:21.499239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ca8dce5b3a4bfdb2648a44e0cac17f)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:21.502102 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:21.674826 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ca8dce5b3a4bfdb2648a44e0cac17f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.757299 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ca8dce5b3a4bfdb2648a44e0cac17f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.772634 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ca8dce5b3a4bfdb2648a44e0cac17f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.779896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ca8dce5b3a4bfdb2648a44e0cac17f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.780631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-f3511ca8-da5d-4bbb-8274-d833b782677d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ca8dce5b3a4bfdb2648a44e0cac17f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.786897 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1291/2609] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:21 2022] DELETE /identity/v3/users/f4ca8dce5b3a4bfdb2648a44e0cac17f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:22.193968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:22.200290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:22.215138 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1677ea1-fde1-4811-adc3-cc4f91d74cba admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:22.287426 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:22.288109 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:22.288394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:22.288673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] PATH_INFO: `/v3/projects/7ff47887cb9841cca42c4f7caf8490cb` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:22.289839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff47887cb9841cca42c4f7caf8490cb)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:22.291780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:22.420708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:22.427678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff47887cb9841cca42c4f7caf8490cb', 'request_id': 'req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.433174 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff47887cb9841cca42c4f7caf8490cb', 'request_id': 'req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.552352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff47887cb9841cca42c4f7caf8490cb', 'request_id': 'req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.564137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] The token cache is being invalidate because project 7ff47887cb9841cca42c4f7caf8490cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:22.564434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.565185 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-cc8bf9c1-7f91-4826-aac0-d6f2cc6b396b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.567196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1319/2610] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:22 2022] DELETE /identity/v3/projects/7ff47887cb9841cca42c4f7caf8490cb => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:22.575069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab285-bab7-4be2-bf26-b4ea6bfdb3c0 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:22.575604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab285-bab7-4be2-bf26-b4ea6bfdb3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:22.575713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab285-bab7-4be2-bf26-b4ea6bfdb3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:22.598043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-44bab285-bab7-4be2-bf26-b4ea6bfdb3c0 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:22.599316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-44bab285-bab7-4be2-bf26-b4ea6bfdb3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:22.660890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1292/2611] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:22.728123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713e74e-59a2-4448-b7c4-b47d9543f1df None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:22.728409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713e74e-59a2-4448-b7c4-b47d9543f1df None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:22.728691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713e74e-59a2-4448-b7c4-b47d9543f1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:22.749809 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-c713e74e-59a2-4448-b7c4-b47d9543f1df None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:22.751092 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c713e74e-59a2-4448-b7c4-b47d9543f1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:22.759558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1320/2612] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:22.763745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bab285-bab7-4be2-bf26-b4ea6bfdb3c0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:22.774745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bab285-bab7-4be2-bf26-b4ea6bfdb3c0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:22.779413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42624e6c-8081-419f-8e7c-432715027045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:22.780018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42624e6c-8081-419f-8e7c-432715027045 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:22.780305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42624e6c-8081-419f-8e7c-432715027045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:22.780584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42624e6c-8081-419f-8e7c-432715027045 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:22.781236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42624e6c-8081-419f-8e7c-432715027045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:00:22.781545 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42624e6c-8081-419f-8e7c-432715027045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:22.783226 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42624e6c-8081-419f-8e7c-432715027045 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:22.796040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1293/2613] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:00:22.800427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713e74e-59a2-4448-b7c4-b47d9543f1df None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:22.809863 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713e74e-59a2-4448-b7c4-b47d9543f1df None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:22.814682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3350525-10a8-4102-83d2-6b86349640e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:22.815640 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3350525-10a8-4102-83d2-6b86349640e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:22.815914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3350525-10a8-4102-83d2-6b86349640e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:22.816187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3350525-10a8-4102-83d2-6b86349640e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:22.816850 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3350525-10a8-4102-83d2-6b86349640e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:22.818784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3350525-10a8-4102-83d2-6b86349640e2 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:22.924422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1321/2614] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:22 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:22.928581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42624e6c-8081-419f-8e7c-432715027045 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:22.939248 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42624e6c-8081-419f-8e7c-432715027045 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:22.966871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:22.997407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c06da-4aa8-4893-a7b8-9d54af28e746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:22.998008 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c06da-4aa8-4893-a7b8-9d54af28e746 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:22.998301 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c06da-4aa8-4893-a7b8-9d54af28e746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:22.998592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c06da-4aa8-4893-a7b8-9d54af28e746 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:22.999210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c06da-4aa8-4893-a7b8-9d54af28e746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:23.001302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c06da-4aa8-4893-a7b8-9d54af28e746 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:23.101442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1294/2615] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:22 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:23.105289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3350525-10a8-4102-83d2-6b86349640e2 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:23.116727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3350525-10a8-4102-83d2-6b86349640e2 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:23.121427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95fdd6-84c6-441b-bb27-5611b4b86484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:23.122029 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95fdd6-84c6-441b-bb27-5611b4b86484 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:23.122312 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95fdd6-84c6-441b-bb27-5611b4b86484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:23.122621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95fdd6-84c6-441b-bb27-5611b4b86484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:23.123268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95fdd6-84c6-441b-bb27-5611b4b86484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:23.124874 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95fdd6-84c6-441b-bb27-5611b4b86484 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:23.135336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1322/2616] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:23.139392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c06da-4aa8-4893-a7b8-9d54af28e746 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:23.150058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c06da-4aa8-4893-a7b8-9d54af28e746 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:23.154726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce800117-9f92-4ad9-9a90-92e0a48d03a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:23.155326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce800117-9f92-4ad9-9a90-92e0a48d03a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:23.155612 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce800117-9f92-4ad9-9a90-92e0a48d03a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:23.155888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce800117-9f92-4ad9-9a90-92e0a48d03a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:23.156532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce800117-9f92-4ad9-9a90-92e0a48d03a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:23.158133 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce800117-9f92-4ad9-9a90-92e0a48d03a9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:23.169665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1295/2617] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:23.173732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95fdd6-84c6-441b-bb27-5611b4b86484 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:23.184332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95fdd6-84c6-441b-bb27-5611b4b86484 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:23.188922 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe7122-17b5-4925-b5c3-0bf259989a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:23.189568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7122-17b5-4925-b5c3-0bf259989a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:23.189852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7122-17b5-4925-b5c3-0bf259989a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:23.190135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7122-17b5-4925-b5c3-0bf259989a3a admin admin] PATH_INFO: `/v3/projects/6331cbb4cc6e42ab963bdd4654e4ec18/users/7d02f0a5d5ab4c1a959f9473e0d6d765/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:23.205047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe7122-17b5-4925-b5c3-0bf259989a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6331cbb4cc6e42ab963bdd4654e4ec18, user_id=7d02f0a5d5ab4c1a959f9473e0d6d765, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:23.208381 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe7122-17b5-4925-b5c3-0bf259989a3a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:23.278214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1323/2618] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:23 2022] PUT /identity/v3/projects/6331cbb4cc6e42ab963bdd4654e4ec18/users/7d02f0a5d5ab4c1a959f9473e0d6d765/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:23.291087 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd6002-9f01-44cd-9e4f-9cc8a39db287 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:23.291382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd6002-9f01-44cd-9e4f-9cc8a39db287 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:23.291689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd6002-9f01-44cd-9e4f-9cc8a39db287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:23.321509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-23dd6002-9f01-44cd-9e4f-9cc8a39db287 None None] MFA Rules not processed for user `7d02f0a5d5ab4c1a959f9473e0d6d765`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:23.322761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-23dd6002-9f01-44cd-9e4f-9cc8a39db287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:23.391050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1296/2619] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:23.402592 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe7122-17b5-4925-b5c3-0bf259989a3a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:23.404293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-93fe7122-17b5-4925-b5c3-0bf259989a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:23.462293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe7122-17b5-4925-b5c3-0bf259989a3a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:23.465937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e976a-4eb9-4030-884e-071d5c25f241 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:23.466575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e976a-4eb9-4030-884e-071d5c25f241 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:23.466860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e976a-4eb9-4030-884e-071d5c25f241 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:23.467140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e976a-4eb9-4030-884e-071d5c25f241 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:23.475944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e976a-4eb9-4030-884e-071d5c25f241 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:23.477387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e976a-4eb9-4030-884e-071d5c25f241 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:23.486192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1324/2620] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:23.507407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd6002-9f01-44cd-9e4f-9cc8a39db287 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:23.569878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd6002-9f01-44cd-9e4f-9cc8a39db287 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:23.574463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:23.575116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:23.575623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:23.575734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] PATH_INFO: `/v3/users/7d02f0a5d5ab4c1a959f9473e0d6d765` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:23.576815 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d02f0a5d5ab4c1a959f9473e0d6d765)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:23.578866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:23.748581 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:23.755800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d02f0a5d5ab4c1a959f9473e0d6d765'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:23.876670 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d02f0a5d5ab4c1a959f9473e0d6d765'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:23.889014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d02f0a5d5ab4c1a959f9473e0d6d765'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:23.897120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d02f0a5d5ab4c1a959f9473e0d6d765'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:23.897869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d02f0a5d5ab4c1a959f9473e0d6d765'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:23.904064 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1297/2621] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:23 2022] DELETE /identity/v3/users/7d02f0a5d5ab4c1a959f9473e0d6d765 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:23.908757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f80b2-ff3f-43e2-9162-fe5ec7aa60b5 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:23.909538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f80b2-ff3f-43e2-9162-fe5ec7aa60b5 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:23.909839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f80b2-ff3f-43e2-9162-fe5ec7aa60b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:23.929134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-f71f80b2-ff3f-43e2-9162-fe5ec7aa60b5 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:23.930384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f71f80b2-ff3f-43e2-9162-fe5ec7aa60b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:23.989148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1325/2622] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:23.999430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:24.001123 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:24.080268 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c61a11-6645-4ba4-a60b-9635b8a81c9e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:24.084563 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780e16b-8d5e-4d7c-a422-6501ffe2ccc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:24.085170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780e16b-8d5e-4d7c-a422-6501ffe2ccc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:24.085457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780e16b-8d5e-4d7c-a422-6501ffe2ccc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:24.085734 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780e16b-8d5e-4d7c-a422-6501ffe2ccc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:24.094805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780e16b-8d5e-4d7c-a422-6501ffe2ccc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:24.096258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780e16b-8d5e-4d7c-a422-6501ffe2ccc8 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:24.106664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1298/2623] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:24.587621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f80b2-ff3f-43e2-9162-fe5ec7aa60b5 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:24.589089 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f71f80b2-ff3f-43e2-9162-fe5ec7aa60b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:24.604973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f80b2-ff3f-43e2-9162-fe5ec7aa60b5 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:24.616346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:24.616346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:24.616346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:24.616346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] PATH_INFO: `/v3/projects/6331cbb4cc6e42ab963bdd4654e4ec18` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:24.616741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6331cbb4cc6e42ab963bdd4654e4ec18)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:24.616741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:24.618929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:24.764258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6331cbb4cc6e42ab963bdd4654e4ec18', 'request_id': 'req-34a48d57-584d-473d-9c19-1482f5c0937c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:24.770671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6331cbb4cc6e42ab963bdd4654e4ec18', 'request_id': 'req-34a48d57-584d-473d-9c19-1482f5c0937c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:24.842907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6331cbb4cc6e42ab963bdd4654e4ec18', 'request_id': 'req-34a48d57-584d-473d-9c19-1482f5c0937c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:24.854376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] The token cache is being invalidate because project 6331cbb4cc6e42ab963bdd4654e4ec18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:24.854705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:24.859854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-34a48d57-584d-473d-9c19-1482f5c0937c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:24.859854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1326/2624] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:24 2022] DELETE /identity/v3/projects/6331cbb4cc6e42ab963bdd4654e4ec18 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:24.861780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b507b6-6233-45c9-afa0-301d4701b703 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:24.862804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b507b6-6233-45c9-afa0-301d4701b703 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:24.863093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b507b6-6233-45c9-afa0-301d4701b703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:24.883589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-29b507b6-6233-45c9-afa0-301d4701b703 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:24.884806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-29b507b6-6233-45c9-afa0-301d4701b703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:24.945322 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1299/2625] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:24.953206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0682d-1e97-4ad8-b20e-f9e2864d5cdd None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:24.953499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0682d-1e97-4ad8-b20e-f9e2864d5cdd None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:24.953783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0682d-1e97-4ad8-b20e-f9e2864d5cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:24.974391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-54a0682d-1e97-4ad8-b20e-f9e2864d5cdd None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:24.975708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-54a0682d-1e97-4ad8-b20e-f9e2864d5cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:24.983908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1327/2626] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:24.989364 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b507b6-6233-45c9-afa0-301d4701b703 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:24.999300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b507b6-6233-45c9-afa0-301d4701b703 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:25.005458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed6e5c-db56-4ac1-8b2b-46eb8a54a2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:25.006065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed6e5c-db56-4ac1-8b2b-46eb8a54a2ce admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:25.006346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed6e5c-db56-4ac1-8b2b-46eb8a54a2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:25.006655 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed6e5c-db56-4ac1-8b2b-46eb8a54a2ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:25.007308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed6e5c-db56-4ac1-8b2b-46eb8a54a2ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:00:25.007609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed6e5c-db56-4ac1-8b2b-46eb8a54a2ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:25.009241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed6e5c-db56-4ac1-8b2b-46eb8a54a2ce admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:25.021736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1300/2627] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:00:25.026258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0682d-1e97-4ad8-b20e-f9e2864d5cdd None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:25.036506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0682d-1e97-4ad8-b20e-f9e2864d5cdd None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:25.041128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c32ba-e6b7-4c44-b424-0c771d3c8740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:25.041724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c32ba-e6b7-4c44-b424-0c771d3c8740 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:25.042000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c32ba-e6b7-4c44-b424-0c771d3c8740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:25.042268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c32ba-e6b7-4c44-b424-0c771d3c8740 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:25.042931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c32ba-e6b7-4c44-b424-0c771d3c8740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:25.044827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c32ba-e6b7-4c44-b424-0c771d3c8740 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:25.124008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1328/2628] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:25 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:25.129324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed6e5c-db56-4ac1-8b2b-46eb8a54a2ce admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:25.139534 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed6e5c-db56-4ac1-8b2b-46eb8a54a2ce admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:25.164993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:25.197561 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1546fc2-3467-4593-9483-7839629d61d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:25.198175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1546fc2-3467-4593-9483-7839629d61d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:25.198586 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1546fc2-3467-4593-9483-7839629d61d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:25.204302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1546fc2-3467-4593-9483-7839629d61d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:25.204302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1546fc2-3467-4593-9483-7839629d61d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:25.204302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1546fc2-3467-4593-9483-7839629d61d5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:25.292904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1301/2629] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:25 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:25.297121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c32ba-e6b7-4c44-b424-0c771d3c8740 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:25.308524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c32ba-e6b7-4c44-b424-0c771d3c8740 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:25.313221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50069308-4972-4821-b24b-5e9a70e8369a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:25.313879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50069308-4972-4821-b24b-5e9a70e8369a admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:25.314159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50069308-4972-4821-b24b-5e9a70e8369a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:25.314430 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50069308-4972-4821-b24b-5e9a70e8369a admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:25.315123 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50069308-4972-4821-b24b-5e9a70e8369a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:25.316745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50069308-4972-4821-b24b-5e9a70e8369a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:25.326982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1329/2630] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:25.331476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1546fc2-3467-4593-9483-7839629d61d5 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:25.341507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1546fc2-3467-4593-9483-7839629d61d5 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:25.346132 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb246b-9c89-44d3-bc05-4e43c13e6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:25.347283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb246b-9c89-44d3-bc05-4e43c13e6c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:25.347569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb246b-9c89-44d3-bc05-4e43c13e6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:25.347841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb246b-9c89-44d3-bc05-4e43c13e6c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:25.348482 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb246b-9c89-44d3-bc05-4e43c13e6c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:25.350078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb246b-9c89-44d3-bc05-4e43c13e6c4c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:25.360136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1302/2631] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:25.365627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50069308-4972-4821-b24b-5e9a70e8369a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:25.375844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50069308-4972-4821-b24b-5e9a70e8369a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:25.380972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e10a1-7ecf-4cc5-bf31-2b4384af9419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:25.381611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e10a1-7ecf-4cc5-bf31-2b4384af9419 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:25.381889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e10a1-7ecf-4cc5-bf31-2b4384af9419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:25.382159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e10a1-7ecf-4cc5-bf31-2b4384af9419 admin admin] PATH_INFO: `/v3/projects/f4b69427fe5e4818be08acc5dcf2561a/users/e83431d4b81a49aea66e223a3915de14/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:25.397151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e10a1-7ecf-4cc5-bf31-2b4384af9419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b69427fe5e4818be08acc5dcf2561a, user_id=e83431d4b81a49aea66e223a3915de14, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:25.399842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e10a1-7ecf-4cc5-bf31-2b4384af9419 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:25.460956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1330/2632] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:25 2022] PUT /identity/v3/projects/f4b69427fe5e4818be08acc5dcf2561a/users/e83431d4b81a49aea66e223a3915de14/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:25.476091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38412a-189b-4822-8bb5-3a31977685ce None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:25.476384 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38412a-189b-4822-8bb5-3a31977685ce None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:25.476664 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38412a-189b-4822-8bb5-3a31977685ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:25.505878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2f38412a-189b-4822-8bb5-3a31977685ce None None] MFA Rules not processed for user `e83431d4b81a49aea66e223a3915de14`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:25.507126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2f38412a-189b-4822-8bb5-3a31977685ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:25.576020 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1303/2633] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:25.588090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e10a1-7ecf-4cc5-bf31-2b4384af9419 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:25.589815 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d83e10a1-7ecf-4cc5-bf31-2b4384af9419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:25.646555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e10a1-7ecf-4cc5-bf31-2b4384af9419 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:25.650099 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceeea39-0119-476a-92f2-72c8bb217ce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:25.650731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceeea39-0119-476a-92f2-72c8bb217ce6 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:25.651012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceeea39-0119-476a-92f2-72c8bb217ce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:25.651282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceeea39-0119-476a-92f2-72c8bb217ce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:25.659712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceeea39-0119-476a-92f2-72c8bb217ce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:25.661137 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceeea39-0119-476a-92f2-72c8bb217ce6 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:25.670084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1331/2634] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:35.572871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=54,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:35.665911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:42.314599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38412a-189b-4822-8bb5-3a31977685ce None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:42.379470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38412a-189b-4822-8bb5-3a31977685ce None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:42.384251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:42.384913 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:42.385209 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:42.385547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] PATH_INFO: `/v3/users/e83431d4b81a49aea66e223a3915de14` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:42.386769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e83431d4b81a49aea66e223a3915de14)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:42.388898 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:42.614891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e83431d4b81a49aea66e223a3915de14'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:42.733585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e83431d4b81a49aea66e223a3915de14'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:42.747739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e83431d4b81a49aea66e223a3915de14'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:42.755233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e83431d4b81a49aea66e223a3915de14'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:42.755948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e83431d4b81a49aea66e223a3915de14'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:42.762658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1304/2635] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:42 2022] DELETE /identity/v3/users/e83431d4b81a49aea66e223a3915de14 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:42.772210 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c343e-94c0-4c8f-a87f-b49f450182c2 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:42.772730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c343e-94c0-4c8f-a87f-b49f450182c2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:42.772848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c343e-94c0-4c8f-a87f-b49f450182c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:42.796337 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-172c343e-94c0-4c8f-a87f-b49f450182c2 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:42.797568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-172c343e-94c0-4c8f-a87f-b49f450182c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:42.876766 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1332/2636] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:42.887203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:42.888777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:42.961012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e243bf-6446-49b8-8669-088b0dc3eeb1 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:42.966169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a84a4-96ca-4ac4-b778-bf682347726a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:42.966843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a84a4-96ca-4ac4-b778-bf682347726a service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:42.967130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a84a4-96ca-4ac4-b778-bf682347726a service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:42.967537 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a84a4-96ca-4ac4-b778-bf682347726a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:42.978769 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a84a4-96ca-4ac4-b778-bf682347726a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:42.981411 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a84a4-96ca-4ac4-b778-bf682347726a service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:42.991619 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1305/2637] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:43.409068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c343e-94c0-4c8f-a87f-b49f450182c2 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:43.410534 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-172c343e-94c0-4c8f-a87f-b49f450182c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:43.422800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c343e-94c0-4c8f-a87f-b49f450182c2 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:43.427816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:43.428558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:43.428923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:43.429371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] PATH_INFO: `/v3/projects/f4b69427fe5e4818be08acc5dcf2561a` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:43.430679 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b69427fe5e4818be08acc5dcf2561a)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:43.438894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:43.557190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b69427fe5e4818be08acc5dcf2561a', 'request_id': 'req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:43.563269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b69427fe5e4818be08acc5dcf2561a', 'request_id': 'req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:43.642193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b69427fe5e4818be08acc5dcf2561a', 'request_id': 'req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:43.654043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] The token cache is being invalidate because project f4b69427fe5e4818be08acc5dcf2561a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:43.654335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:43.655009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b075e8d0-0a3c-47ac-80ab-5ff7ff19da66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:43.656651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1333/2638] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:43 2022] DELETE /identity/v3/projects/f4b69427fe5e4818be08acc5dcf2561a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:43.661706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e178c-4470-40e1-948a-b73bf0771732 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:43.662213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e178c-4470-40e1-948a-b73bf0771732 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:43.662326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e178c-4470-40e1-948a-b73bf0771732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:43.681868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-795e178c-4470-40e1-948a-b73bf0771732 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:43.683812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-795e178c-4470-40e1-948a-b73bf0771732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:43.701467 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:43.743869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1306/2639] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:00:43.753524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408ff70-6267-413c-bb60-fcd434faf8e9 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:43.753816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408ff70-6267-413c-bb60-fcd434faf8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:43.754106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408ff70-6267-413c-bb60-fcd434faf8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:43.774710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-7408ff70-6267-413c-bb60-fcd434faf8e9 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:43.775929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7408ff70-6267-413c-bb60-fcd434faf8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:43.784307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1334/2640] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:43.788690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e178c-4470-40e1-948a-b73bf0771732 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:43.799506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e178c-4470-40e1-948a-b73bf0771732 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:43.805420 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82722c-380d-454a-be7e-4fb369306df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:43.806067 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82722c-380d-454a-be7e-4fb369306df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:43.806613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82722c-380d-454a-be7e-4fb369306df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:43.807106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82722c-380d-454a-be7e-4fb369306df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:43.808961 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82722c-380d-454a-be7e-4fb369306df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:00:43.809508 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82722c-380d-454a-be7e-4fb369306df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:43.811374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82722c-380d-454a-be7e-4fb369306df6 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:43.825493 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1307/2641] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:00:43.830499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408ff70-6267-413c-bb60-fcd434faf8e9 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:43.840470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408ff70-6267-413c-bb60-fcd434faf8e9 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:43.845261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46911f6-6450-45c8-96e1-da6c3bd975cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:43.845865 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46911f6-6450-45c8-96e1-da6c3bd975cc admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:43.846145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46911f6-6450-45c8-96e1-da6c3bd975cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:43.846424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46911f6-6450-45c8-96e1-da6c3bd975cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:43.847095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46911f6-6450-45c8-96e1-da6c3bd975cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:43.849008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46911f6-6450-45c8-96e1-da6c3bd975cc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:43.987336 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1335/2642] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:43 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:43.991453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82722c-380d-454a-be7e-4fb369306df6 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:44.002847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82722c-380d-454a-be7e-4fb369306df6 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:44.062159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3903d90-bcef-43c4-ad6c-1f4b1bf7035d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:44.062823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3903d90-bcef-43c4-ad6c-1f4b1bf7035d admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:44.063086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3903d90-bcef-43c4-ad6c-1f4b1bf7035d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:44.063361 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3903d90-bcef-43c4-ad6c-1f4b1bf7035d admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:44.063981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3903d90-bcef-43c4-ad6c-1f4b1bf7035d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:44.065912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3903d90-bcef-43c4-ad6c-1f4b1bf7035d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:44.202568 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1308/2643] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:43 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:44.207332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46911f6-6450-45c8-96e1-da6c3bd975cc admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:44.217622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46911f6-6450-45c8-96e1-da6c3bd975cc admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:44.222268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04311a8d-b907-42db-9f4a-d4b99f93cd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:44.222909 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04311a8d-b907-42db-9f4a-d4b99f93cd85 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:44.223196 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04311a8d-b907-42db-9f4a-d4b99f93cd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:44.223474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04311a8d-b907-42db-9f4a-d4b99f93cd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:44.224133 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04311a8d-b907-42db-9f4a-d4b99f93cd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:44.225751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04311a8d-b907-42db-9f4a-d4b99f93cd85 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:44.236750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1336/2644] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:44.240881 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3903d90-bcef-43c4-ad6c-1f4b1bf7035d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:44.251249 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3903d90-bcef-43c4-ad6c-1f4b1bf7035d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:44.256246 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940071dd-a62b-4630-b1a4-7527b80b2025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:44.256842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940071dd-a62b-4630-b1a4-7527b80b2025 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:44.257203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940071dd-a62b-4630-b1a4-7527b80b2025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:44.257478 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940071dd-a62b-4630-b1a4-7527b80b2025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:44.258208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940071dd-a62b-4630-b1a4-7527b80b2025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:44.259851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940071dd-a62b-4630-b1a4-7527b80b2025 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:44.270474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1309/2645] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:44.276035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04311a8d-b907-42db-9f4a-d4b99f93cd85 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:44.285130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04311a8d-b907-42db-9f4a-d4b99f93cd85 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:44.289742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050696a-675f-458a-b9e0-8b5aaf954d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:44.290405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050696a-675f-458a-b9e0-8b5aaf954d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:44.290709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050696a-675f-458a-b9e0-8b5aaf954d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:44.290985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050696a-675f-458a-b9e0-8b5aaf954d50 admin admin] PATH_INFO: `/v3/projects/3b064eb9c6944e11b79e161e4df5205a/users/8cfff1e957f1464dbee571606dc39624/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:44.306263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050696a-675f-458a-b9e0-8b5aaf954d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b064eb9c6944e11b79e161e4df5205a, user_id=8cfff1e957f1464dbee571606dc39624, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:44.310644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050696a-675f-458a-b9e0-8b5aaf954d50 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:44.403852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1337/2646] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:44 2022] PUT /identity/v3/projects/3b064eb9c6944e11b79e161e4df5205a/users/8cfff1e957f1464dbee571606dc39624/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:44.408674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1b088-569f-43ef-868e-9f20be569dcd None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:44.408971 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1b088-569f-43ef-868e-9f20be569dcd None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:44.409256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1b088-569f-43ef-868e-9f20be569dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:44.429479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-9af1b088-569f-43ef-868e-9f20be569dcd None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:44.430091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9af1b088-569f-43ef-868e-9f20be569dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:44.494431 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1310/2647] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:44.504700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050696a-675f-458a-b9e0-8b5aaf954d50 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:44.507304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9050696a-675f-458a-b9e0-8b5aaf954d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:44.533779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:44.570973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050696a-675f-458a-b9e0-8b5aaf954d50 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:44.575122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f4318-e71a-42a8-8922-d0210499de81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:44.575727 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f4318-e71a-42a8-8922-d0210499de81 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:44.576021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f4318-e71a-42a8-8922-d0210499de81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:44.576291 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f4318-e71a-42a8-8922-d0210499de81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:44.585286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f4318-e71a-42a8-8922-d0210499de81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:44.586751 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f4318-e71a-42a8-8922-d0210499de81 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:44.597187 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1338/2648] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:45.907498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca8e49-2f93-42db-8973-53f8ad656811 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:45.908163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca8e49-2f93-42db-8973-53f8ad656811 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:45.908163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca8e49-2f93-42db-8973-53f8ad656811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:45.942787 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-6eca8e49-2f93-42db-8973-53f8ad656811 None None] MFA Rules not processed for user `8cfff1e957f1464dbee571606dc39624`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:45.944154 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6eca8e49-2f93-42db-8973-53f8ad656811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:45.961269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:46.013377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1311/2649] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:46.030806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f4318-e71a-42a8-8922-d0210499de81 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:46.032408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-324f4318-e71a-42a8-8922-d0210499de81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:46.122528 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f4318-e71a-42a8-8922-d0210499de81 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:46.126259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66045bab-9e71-4b46-8ce0-ff059a1ebcc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:46.126913 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045bab-9e71-4b46-8ce0-ff059a1ebcc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:46.127197 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045bab-9e71-4b46-8ce0-ff059a1ebcc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:46.127729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045bab-9e71-4b46-8ce0-ff059a1ebcc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:46.136673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66045bab-9e71-4b46-8ce0-ff059a1ebcc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:46.138122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66045bab-9e71-4b46-8ce0-ff059a1ebcc6 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:46.147078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1339/2650] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:52.060311 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f70d0-f44f-4d97-b241-284468c14637 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:52.060864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f70d0-f44f-4d97-b241-284468c14637 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:52.060864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f70d0-f44f-4d97-b241-284468c14637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:52.080748 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-479f70d0-f44f-4d97-b241-284468c14637 None None] MFA Rules not processed for user `8cfff1e957f1464dbee571606dc39624`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:52.081942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-479f70d0-f44f-4d97-b241-284468c14637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:52.088795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1312/2651] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:54.777134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66045bab-9e71-4b46-8ce0-ff059a1ebcc6 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:54.778698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-66045bab-9e71-4b46-8ce0-ff059a1ebcc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:54.792246 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66045bab-9e71-4b46-8ce0-ff059a1ebcc6 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:54.797038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:54.797701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:54.798055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:54.798632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] PATH_INFO: `/v3/users/62db2c233baf42b796523ca625e207c7` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:54.799518 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62db2c233baf42b796523ca625e207c7)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:54.801459 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:54.967322 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62db2c233baf42b796523ca625e207c7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:54.967996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62db2c233baf42b796523ca625e207c7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:55.067779 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f70d0-f44f-4d97-b241-284468c14637 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:55.069316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-479f70d0-f44f-4d97-b241-284468c14637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:55.069729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62db2c233baf42b796523ca625e207c7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:55.080434 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62db2c233baf42b796523ca625e207c7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:55.095218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-33adcb58-acee-4cdf-9a98-6f3df26bcb90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62db2c233baf42b796523ca625e207c7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:55.102944 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1340/2652] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:54 2022] DELETE /identity/v3/users/62db2c233baf42b796523ca625e207c7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:55.108366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381839-28e9-4807-9a82-f31cefa6a8e8 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:55.108667 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381839-28e9-4807-9a82-f31cefa6a8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:55.108980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381839-28e9-4807-9a82-f31cefa6a8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:55.132063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-57381839-28e9-4807-9a82-f31cefa6a8e8 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:55.133355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-57381839-28e9-4807-9a82-f31cefa6a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:55.149094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:55.206173 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1341/2653] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:00:55.216837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381839-28e9-4807-9a82-f31cefa6a8e8 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:55.219408 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-57381839-28e9-4807-9a82-f31cefa6a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:55.288541 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381839-28e9-4807-9a82-f31cefa6a8e8 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:55.294075 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4f77f-b227-4396-abfc-fe31860cf1fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:55.294662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4f77f-b227-4396-abfc-fe31860cf1fc service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:55.295061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4f77f-b227-4396-abfc-fe31860cf1fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:55.295236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4f77f-b227-4396-abfc-fe31860cf1fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:55.317990 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4f77f-b227-4396-abfc-fe31860cf1fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:55.319523 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4f77f-b227-4396-abfc-fe31860cf1fc service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:55.331964 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f70d0-f44f-4d97-b241-284468c14637 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:55.335438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1342/2654] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:00:55.422844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d955c9-3464-47ed-a897-bd34df4f082c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:55.423594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d955c9-3464-47ed-a897-bd34df4f082c service placement] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:55.423869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d955c9-3464-47ed-a897-bd34df4f082c service placement] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:55.424153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d955c9-3464-47ed-a897-bd34df4f082c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:55.427558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-73d955c9-3464-47ed-a897-bd34df4f082c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:55.443784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d955c9-3464-47ed-a897-bd34df4f082c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:55.445426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d955c9-3464-47ed-a897-bd34df4f082c service placement] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:55.456524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1313/2655] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:00:56.334746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4f77f-b227-4396-abfc-fe31860cf1fc service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:56.336675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-84b4f77f-b227-4396-abfc-fe31860cf1fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:56.350387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4f77f-b227-4396-abfc-fe31860cf1fc service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:56.355213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:56.355879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:56.356160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:56.356437 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] PATH_INFO: `/v3/projects/fc9d7ceca2474de39949bea6bbe0c2d0` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:56.357567 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc9d7ceca2474de39949bea6bbe0c2d0)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:56.359631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:56.482650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc9d7ceca2474de39949bea6bbe0c2d0', 'request_id': 'req-c9db2a71-02d7-4237-9acd-0a4485438233'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:56.488275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc9d7ceca2474de39949bea6bbe0c2d0', 'request_id': 'req-c9db2a71-02d7-4237-9acd-0a4485438233'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:56.584471 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc9d7ceca2474de39949bea6bbe0c2d0', 'request_id': 'req-c9db2a71-02d7-4237-9acd-0a4485438233'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:56.597702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] The token cache is being invalidate because project fc9d7ceca2474de39949bea6bbe0c2d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:56.597993 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:56.598691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:56.600256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1343/2656] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:56 2022] DELETE /identity/v3/projects/fc9d7ceca2474de39949bea6bbe0c2d0 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:56.604914 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d955c9-3464-47ed-a897-bd34df4f082c service placement] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:56.615620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d955c9-3464-47ed-a897-bd34df4f082c service placement] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:56.637415 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:56.674376 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:56.675204 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:56.675527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:56.675761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] PATH_INFO: `/v3/users/c2bc2801e9d3479fa8a41465b73c3928` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:56.676890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2bc2801e9d3479fa8a41465b73c3928)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:56.678853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:56.731025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:56.732569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:56.747194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db2a71-02d7-4237-9acd-0a4485438233 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:56.752130 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:56.753040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:56.753303 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:56.753575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] PATH_INFO: `/v3/users/59ce5bae5ff1444883b78f78abda74ea` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:56.754927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] RBAC: Authorizing `identity:delete_user(user_id=59ce5bae5ff1444883b78f78abda74ea)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:56.756842 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:56.941831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2bc2801e9d3479fa8a41465b73c3928'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:56.960692 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:56.971004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59ce5bae5ff1444883b78f78abda74ea'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:56.971004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59ce5bae5ff1444883b78f78abda74ea'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.060601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2bc2801e9d3479fa8a41465b73c3928'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.073014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2bc2801e9d3479fa8a41465b73c3928'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.090251 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2bc2801e9d3479fa8a41465b73c3928'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.098032 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-e11f5107-e7bf-4c89-9a7e-1bba551f3ea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2bc2801e9d3479fa8a41465b73c3928'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.106604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1314/2657] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:56 2022] DELETE /identity/v3/users/c2bc2801e9d3479fa8a41465b73c3928 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:57.144639 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59ce5bae5ff1444883b78f78abda74ea'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.153685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59ce5bae5ff1444883b78f78abda74ea'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.170732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59ce5bae5ff1444883b78f78abda74ea'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.184634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1344/2658] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:56 2022] DELETE /identity/v3/users/59ce5bae5ff1444883b78f78abda74ea => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:00:57.189302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d69478-0b3d-468c-bf21-ab106739ee85 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:57.189521 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d69478-0b3d-468c-bf21-ab106739ee85 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:57.189802 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d69478-0b3d-468c-bf21-ab106739ee85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:57.212524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d2d69478-0b3d-468c-bf21-ab106739ee85 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:57.214476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d2d69478-0b3d-468c-bf21-ab106739ee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:57.286643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1315/2659] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:57.295221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:57.296681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:57.382568 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6017e2-5927-444f-aec4-1a6e7f45313e admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:57.387372 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32e2d9-ab64-4dfe-9d3b-a75f5e0ef996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:57.388008 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e2d9-ab64-4dfe-9d3b-a75f5e0ef996 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:57.388442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e2d9-ab64-4dfe-9d3b-a75f5e0ef996 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:57.392209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e2d9-ab64-4dfe-9d3b-a75f5e0ef996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:57.399411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32e2d9-ab64-4dfe-9d3b-a75f5e0ef996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:57.401687 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32e2d9-ab64-4dfe-9d3b-a75f5e0ef996 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:57.413319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1345/2660] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:57.710084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d69478-0b3d-468c-bf21-ab106739ee85 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:57.712501 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d2d69478-0b3d-468c-bf21-ab106739ee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:57.729203 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d69478-0b3d-468c-bf21-ab106739ee85 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:57.734068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:57.734891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:57.735177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:57.735459 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] PATH_INFO: `/v3/projects/74a63f91b65c4a91b76190695f26a05c` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:57.736633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a63f91b65c4a91b76190695f26a05c)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:57.738767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:57.861178 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32e2d9-ab64-4dfe-9d3b-a75f5e0ef996 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:57.862917 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2d32e2d9-ab64-4dfe-9d3b-a75f5e0ef996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:57.883021 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:57.884100 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32e2d9-ab64-4dfe-9d3b-a75f5e0ef996 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:57.891221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a63f91b65c4a91b76190695f26a05c', 'request_id': 'req-d9121da9-8c92-4574-812e-92cc8444dc1f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.955778 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:57.956550 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:57.956929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:57.957221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] PATH_INFO: `/v3/projects/cb1a69cce5cd4a8393d04fdcbe9dea6f` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:57.958581 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1a69cce5cd4a8393d04fdcbe9dea6f)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:57.960517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:57.974125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a63f91b65c4a91b76190695f26a05c', 'request_id': 'req-d9121da9-8c92-4574-812e-92cc8444dc1f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.980683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a63f91b65c4a91b76190695f26a05c', 'request_id': 'req-d9121da9-8c92-4574-812e-92cc8444dc1f'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.992283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] The token cache is being invalidate because project 74a63f91b65c4a91b76190695f26a05c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:57.992572 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.993221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d9121da9-8c92-4574-812e-92cc8444dc1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:57.995549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1316/2661] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:57 2022] DELETE /identity/v3/projects/74a63f91b65c4a91b76190695f26a05c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:58.000515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b14a8-f905-4e7e-9272-a6f8e16c3bc9 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:58.000800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b14a8-f905-4e7e-9272-a6f8e16c3bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:58.001177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b14a8-f905-4e7e-9272-a6f8e16c3bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:58.025650 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-8f2b14a8-f905-4e7e-9272-a6f8e16c3bc9 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:58.027057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8f2b14a8-f905-4e7e-9272-a6f8e16c3bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:58.035586 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1317/2662] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:00:58.044175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fefde-168d-46a1-9458-3e91b23c0615 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:58.044464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fefde-168d-46a1-9458-3e91b23c0615 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:58.044744 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fefde-168d-46a1-9458-3e91b23c0615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:58.069525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-652fefde-168d-46a1-9458-3e91b23c0615 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:58.071038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-652fefde-168d-46a1-9458-3e91b23c0615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:58.083354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1a69cce5cd4a8393d04fdcbe9dea6f', 'request_id': 'req-4031ab18-95f9-4858-ba92-ff0895dce9fe'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:58.091822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1a69cce5cd4a8393d04fdcbe9dea6f', 'request_id': 'req-4031ab18-95f9-4858-ba92-ff0895dce9fe'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:58.094891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:58.143869 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1318/2663] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:00:58.150170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fefde-168d-46a1-9458-3e91b23c0615 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:58.160097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fefde-168d-46a1-9458-3e91b23c0615 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:58.164866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533702c5-0451-4e96-86e6-e37f21c2841e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:58.165691 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533702c5-0451-4e96-86e6-e37f21c2841e admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:58.166710 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533702c5-0451-4e96-86e6-e37f21c2841e admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:58.167772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533702c5-0451-4e96-86e6-e37f21c2841e admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:58.168444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533702c5-0451-4e96-86e6-e37f21c2841e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:00:58.168751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533702c5-0451-4e96-86e6-e37f21c2841e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:58.170360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533702c5-0451-4e96-86e6-e37f21c2841e admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:58.179040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1a69cce5cd4a8393d04fdcbe9dea6f', 'request_id': 'req-4031ab18-95f9-4858-ba92-ff0895dce9fe'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:58.187866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1319/2664] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:00:58.189782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533702c5-0451-4e96-86e6-e37f21c2841e admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:58.200213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] The token cache is being invalidate because project cb1a69cce5cd4a8393d04fdcbe9dea6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:58.200515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:58.201354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:58.212327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1346/2665] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:57 2022] DELETE /identity/v3/projects/cb1a69cce5cd4a8393d04fdcbe9dea6f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:58.212552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533702c5-0451-4e96-86e6-e37f21c2841e admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:58.212552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7f180-b7eb-480f-be4c-79e6aee2005d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:58.212552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7f180-b7eb-480f-be4c-79e6aee2005d admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:58.212552 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7f180-b7eb-480f-be4c-79e6aee2005d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:58.216873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7f180-b7eb-480f-be4c-79e6aee2005d admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:58.216873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7f180-b7eb-480f-be4c-79e6aee2005d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:58.216873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7f180-b7eb-480f-be4c-79e6aee2005d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:58.370984 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1320/2666] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:58 2022] POST /identity/v3/projects => generated 381 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:58.374954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:58.376622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:58.391457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031ab18-95f9-4858-ba92-ff0895dce9fe admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:58.476304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f58b97-86a3-4733-87ce-9e99bc761966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:58.476933 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58b97-86a3-4733-87ce-9e99bc761966 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:58.477213 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58b97-86a3-4733-87ce-9e99bc761966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:58.477482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58b97-86a3-4733-87ce-9e99bc761966 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:58.478140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f58b97-86a3-4733-87ce-9e99bc761966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:58.481046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f58b97-86a3-4733-87ce-9e99bc761966 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:58.595236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1347/2667] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:58 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:58.596726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7f180-b7eb-480f-be4c-79e6aee2005d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:58.608637 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7f180-b7eb-480f-be4c-79e6aee2005d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:58.613993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b678e-9ede-46b0-962c-ec1ab8e6838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:58.614777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b678e-9ede-46b0-962c-ec1ab8e6838a admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:58.615066 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b678e-9ede-46b0-962c-ec1ab8e6838a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:58.615350 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b678e-9ede-46b0-962c-ec1ab8e6838a admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:58.616036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b678e-9ede-46b0-962c-ec1ab8e6838a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:58.618065 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b678e-9ede-46b0-962c-ec1ab8e6838a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:58.634360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1321/2668] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:58.638875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f58b97-86a3-4733-87ce-9e99bc761966 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:58.654037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f58b97-86a3-4733-87ce-9e99bc761966 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:58.659083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b436f1-087d-4442-ade2-93891dcbad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:58.659703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b436f1-087d-4442-ade2-93891dcbad08 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:58.660165 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b436f1-087d-4442-ade2-93891dcbad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:58.660452 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b436f1-087d-4442-ade2-93891dcbad08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:58.661105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b436f1-087d-4442-ade2-93891dcbad08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:58.662908 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b436f1-087d-4442-ade2-93891dcbad08 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:58.677250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1348/2669] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:58.683370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b678e-9ede-46b0-962c-ec1ab8e6838a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:58.694170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b678e-9ede-46b0-962c-ec1ab8e6838a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:58.699727 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ff17a-5ad5-4aa3-9452-acd87bb159c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:58.700904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ff17a-5ad5-4aa3-9452-acd87bb159c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:58.700904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ff17a-5ad5-4aa3-9452-acd87bb159c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:58.701157 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ff17a-5ad5-4aa3-9452-acd87bb159c1 admin admin] PATH_INFO: `/v3/projects/311cf04ffc0343b580c97526af529c48/users/764786d99a954fa49585f57705ee95c3/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:58.720216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ff17a-5ad5-4aa3-9452-acd87bb159c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=311cf04ffc0343b580c97526af529c48, user_id=764786d99a954fa49585f57705ee95c3, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:58.724514 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ff17a-5ad5-4aa3-9452-acd87bb159c1 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:58.808765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1322/2670] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:58 2022] PUT /identity/v3/projects/311cf04ffc0343b580c97526af529c48/users/764786d99a954fa49585f57705ee95c3/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:00:58.820390 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea55e5c-c7df-47a5-b363-1d329da2abf5 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:58.820682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea55e5c-c7df-47a5-b363-1d329da2abf5 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:58.820962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea55e5c-c7df-47a5-b363-1d329da2abf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:58.851482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-fea55e5c-c7df-47a5-b363-1d329da2abf5 None None] MFA Rules not processed for user `764786d99a954fa49585f57705ee95c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:58.852801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-fea55e5c-c7df-47a5-b363-1d329da2abf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:58.942589 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1349/2671] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:58.948405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ff17a-5ad5-4aa3-9452-acd87bb159c1 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:59.021863 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:59.035158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ff17a-5ad5-4aa3-9452-acd87bb159c1 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:59.040465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129aa62-7e9d-4f14-ac87-430cac481a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:59.041191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129aa62-7e9d-4f14-ac87-430cac481a69 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:59.041576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129aa62-7e9d-4f14-ac87-430cac481a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:59.042057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129aa62-7e9d-4f14-ac87-430cac481a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:59.042754 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129aa62-7e9d-4f14-ac87-430cac481a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:59.046173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129aa62-7e9d-4f14-ac87-430cac481a69 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:59.147715 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1323/2672] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:00:58 2022] POST /identity/v3/projects => generated 381 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:00:59.153060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea55e5c-c7df-47a5-b363-1d329da2abf5 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:59.164072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea55e5c-c7df-47a5-b363-1d329da2abf5 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:59.222859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:59.229211 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a483e99-dea7-415a-a570-f81b52ad2148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:59.229934 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a483e99-dea7-415a-a570-f81b52ad2148 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:59.230192 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a483e99-dea7-415a-a570-f81b52ad2148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:59.230467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a483e99-dea7-415a-a570-f81b52ad2148 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:59.231140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a483e99-dea7-415a-a570-f81b52ad2148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:59.233082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a483e99-dea7-415a-a570-f81b52ad2148 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:59.328557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1350/2673] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:00:59 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:00:59.332583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129aa62-7e9d-4f14-ac87-430cac481a69 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:59.343395 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129aa62-7e9d-4f14-ac87-430cac481a69 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:59.348116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db62fd-a38d-438a-ae5b-b1277ebdde19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:59.348723 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db62fd-a38d-438a-ae5b-b1277ebdde19 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:59.349057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db62fd-a38d-438a-ae5b-b1277ebdde19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:59.349647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db62fd-a38d-438a-ae5b-b1277ebdde19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:59.350269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db62fd-a38d-438a-ae5b-b1277ebdde19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:59.351894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db62fd-a38d-438a-ae5b-b1277ebdde19 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:59.363819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1324/2674] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:59.368121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a483e99-dea7-415a-a570-f81b52ad2148 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:59.379848 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a483e99-dea7-415a-a570-f81b52ad2148 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:59.384619 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9637d-c095-46a3-bd8e-4f484d3a8073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:59.390361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9637d-c095-46a3-bd8e-4f484d3a8073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:59.390361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9637d-c095-46a3-bd8e-4f484d3a8073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:59.390361 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9637d-c095-46a3-bd8e-4f484d3a8073 admin admin] PATH_INFO: `/v3/projects/ddbb72d18d3e46f19f18c88b1382e36c/users/5dc64c33027644b682a67b15cbb92bfa/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:59.402421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9637d-c095-46a3-bd8e-4f484d3a8073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbb72d18d3e46f19f18c88b1382e36c, user_id=5dc64c33027644b682a67b15cbb92bfa, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:59.405145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9637d-c095-46a3-bd8e-4f484d3a8073 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:59.486879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1351/2675] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:59 2022] PUT /identity/v3/projects/ddbb72d18d3e46f19f18c88b1382e36c/users/5dc64c33027644b682a67b15cbb92bfa/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:59.490857 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db62fd-a38d-438a-ae5b-b1277ebdde19 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:59.501924 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db62fd-a38d-438a-ae5b-b1277ebdde19 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:59.560872 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf09ed-4b61-4adc-afd4-3f62df604716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:59.561393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf09ed-4b61-4adc-afd4-3f62df604716 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:59.561674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf09ed-4b61-4adc-afd4-3f62df604716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:59.561948 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf09ed-4b61-4adc-afd4-3f62df604716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:59.562621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf09ed-4b61-4adc-afd4-3f62df604716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:59.564214 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf09ed-4b61-4adc-afd4-3f62df604716 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:59.574244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1325/2676] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:00:59.578457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9637d-c095-46a3-bd8e-4f484d3a8073 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:59.589665 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9637d-c095-46a3-bd8e-4f484d3a8073 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:59.594307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e41873-89be-4c5c-b702-0e731ed70a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:59.594996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e41873-89be-4c5c-b702-0e731ed70a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:59.595278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e41873-89be-4c5c-b702-0e731ed70a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:59.595551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e41873-89be-4c5c-b702-0e731ed70a0d admin admin] PATH_INFO: `/v3/projects/ddbb72d18d3e46f19f18c88b1382e36c/users/5dc64c33027644b682a67b15cbb92bfa/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:59.597460 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e41873-89be-4c5c-b702-0e731ed70a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbb72d18d3e46f19f18c88b1382e36c, user_id=5dc64c33027644b682a67b15cbb92bfa, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:59.600156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e41873-89be-4c5c-b702-0e731ed70a0d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:59.611258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1352/2677] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:00:59 2022] PUT /identity/v3/projects/ddbb72d18d3e46f19f18c88b1382e36c/users/5dc64c33027644b682a67b15cbb92bfa/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:59.630617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94084a10-b95e-45bd-b1a8-63e4ffa9b0e4 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:59.630910 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94084a10-b95e-45bd-b1a8-63e4ffa9b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:59.631196 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94084a10-b95e-45bd-b1a8-63e4ffa9b0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:59.659558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-94084a10-b95e-45bd-b1a8-63e4ffa9b0e4 None None] MFA Rules not processed for user `5dc64c33027644b682a67b15cbb92bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:00:59.663893 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-94084a10-b95e-45bd-b1a8-63e4ffa9b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:59.744860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1326/2678] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:00:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:00:59.757659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e41873-89be-4c5c-b702-0e731ed70a0d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:59.759637 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e3e41873-89be-4c5c-b702-0e731ed70a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:59.823835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e41873-89be-4c5c-b702-0e731ed70a0d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:59.827467 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5fc16-4e0c-4f31-abce-e912c54c8b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:59.828080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5fc16-4e0c-4f31-abce-e912c54c8b2c service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:59.828356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5fc16-4e0c-4f31-abce-e912c54c8b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:59.828624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5fc16-4e0c-4f31-abce-e912c54c8b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:59.838730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5fc16-4e0c-4f31-abce-e912c54c8b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:59.840126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5fc16-4e0c-4f31-abce-e912c54c8b2c service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:59.855425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1353/2679] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:00.078593 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94084a10-b95e-45bd-b1a8-63e4ffa9b0e4 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:00.089095 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94084a10-b95e-45bd-b1a8-63e4ffa9b0e4 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:00.093058 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e00b7-737c-459e-9c62-1264d6533a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:00.093682 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e00b7-737c-459e-9c62-1264d6533a0e service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:00.093962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e00b7-737c-459e-9c62-1264d6533a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:00.094238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e00b7-737c-459e-9c62-1264d6533a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:00.140703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:00.154531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e00b7-737c-459e-9c62-1264d6533a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:00.156007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e00b7-737c-459e-9c62-1264d6533a0e service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:00.165016 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1327/2680] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:09.846867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=44,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:10.470275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=7 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:16.841369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5fc16-4e0c-4f31-abce-e912c54c8b2c service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:16.911383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5fc16-4e0c-4f31-abce-e912c54c8b2c service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:16.915971 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:16.916603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:16.916969 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:16.917152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] PATH_INFO: `/v3/users/764786d99a954fa49585f57705ee95c3` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:16.918299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=764786d99a954fa49585f57705ee95c3)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:16.922450 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:17.149732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764786d99a954fa49585f57705ee95c3'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:17.150454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764786d99a954fa49585f57705ee95c3'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:17.241383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764786d99a954fa49585f57705ee95c3'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:17.249631 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764786d99a954fa49585f57705ee95c3'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:17.262135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764786d99a954fa49585f57705ee95c3'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:17.269519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1354/2681] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:16 2022] DELETE /identity/v3/users/764786d99a954fa49585f57705ee95c3 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:17.273966 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068447a-b2dc-4cd7-88f5-a840a3ca2c0a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:17.274483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068447a-b2dc-4cd7-88f5-a840a3ca2c0a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:17.274676 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068447a-b2dc-4cd7-88f5-a840a3ca2c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:17.293755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d068447a-b2dc-4cd7-88f5-a840a3ca2c0a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:17.295061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d068447a-b2dc-4cd7-88f5-a840a3ca2c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:17.356059 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1328/2682] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:17.366237 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:17.367673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:17.427487 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fbc16-3298-459a-9a01-fbf03c6f32dc admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:17.431088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c2043-2faf-4e02-a11f-53ff06295329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:17.431701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c2043-2faf-4e02-a11f-53ff06295329 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:17.431983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c2043-2faf-4e02-a11f-53ff06295329 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:17.432259 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c2043-2faf-4e02-a11f-53ff06295329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:17.442364 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c2043-2faf-4e02-a11f-53ff06295329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:17.443835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c2043-2faf-4e02-a11f-53ff06295329 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:17.454576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1355/2683] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:17.923701 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068447a-b2dc-4cd7-88f5-a840a3ca2c0a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:17.928116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d068447a-b2dc-4cd7-88f5-a840a3ca2c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:17.949801 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068447a-b2dc-4cd7-88f5-a840a3ca2c0a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:17.954896 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:17.955673 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:17.955821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:17.956098 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] PATH_INFO: `/v3/projects/311cf04ffc0343b580c97526af529c48` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:17.957270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] RBAC: Authorizing `identity:delete_project(project_id=311cf04ffc0343b580c97526af529c48)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:17.959506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:18.076439 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e2751-7bdd-4187-b5e1-1c296bce4759 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:18.077148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e2751-7bdd-4187-b5e1-1c296bce4759 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:18.077148 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e2751-7bdd-4187-b5e1-1c296bce4759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:18.102805 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-1b6e2751-7bdd-4187-b5e1-1c296bce4759 None None] MFA Rules not processed for user `8cfff1e957f1464dbee571606dc39624`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:18.104060 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1b6e2751-7bdd-4187-b5e1-1c296bce4759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:18.118979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '311cf04ffc0343b580c97526af529c48', 'request_id': 'req-081c2c87-26bc-4582-8482-5606239e2fae'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.127858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:18.169451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1356/2684] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:01:18.217708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '311cf04ffc0343b580c97526af529c48', 'request_id': 'req-081c2c87-26bc-4582-8482-5606239e2fae'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.227336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '311cf04ffc0343b580c97526af529c48', 'request_id': 'req-081c2c87-26bc-4582-8482-5606239e2fae'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.242042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] The token cache is being invalidate because project 311cf04ffc0343b580c97526af529c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:18.242342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.243039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.247470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1329/2685] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:17 2022] DELETE /identity/v3/projects/311cf04ffc0343b580c97526af529c48 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:01:18.249524 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e2751-7bdd-4187-b5e1-1c296bce4759 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:18.316496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e2751-7bdd-4187-b5e1-1c296bce4759 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:18.321114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:18.321757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:18.322038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:18.322315 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] PATH_INFO: `/v3/users/5dc64c33027644b682a67b15cbb92bfa` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:18.323490 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc64c33027644b682a67b15cbb92bfa)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:18.325488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:18.556732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc64c33027644b682a67b15cbb92bfa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.557461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc64c33027644b682a67b15cbb92bfa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.869980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc64c33027644b682a67b15cbb92bfa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.878724 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc64c33027644b682a67b15cbb92bfa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.894386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-56ce141b-f109-493e-8f65-1d12a15a021e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc64c33027644b682a67b15cbb92bfa'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:18.901486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1357/2686] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:18 2022] DELETE /identity/v3/users/5dc64c33027644b682a67b15cbb92bfa => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:19.309400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:19.309400 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:19.337679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c2c87-26bc-4582-8482-5606239e2fae admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:19.400223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:19.402402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:19.402402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:19.402402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] PATH_INFO: `/v3/projects/ddbb72d18d3e46f19f18c88b1382e36c` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:19.402743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbb72d18d3e46f19f18c88b1382e36c)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:19.404430 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:19.562916 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:19.579850 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbb72d18d3e46f19f18c88b1382e36c', 'request_id': 'req-5f047fa2-bf34-47f8-baac-0208ff8642ef'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:19.697241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbb72d18d3e46f19f18c88b1382e36c', 'request_id': 'req-5f047fa2-bf34-47f8-baac-0208ff8642ef'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:19.704515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbb72d18d3e46f19f18c88b1382e36c', 'request_id': 'req-5f047fa2-bf34-47f8-baac-0208ff8642ef'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:19.715005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] The token cache is being invalidate because project ddbb72d18d3e46f19f18c88b1382e36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:19.715305 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:19.715978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-5f047fa2-bf34-47f8-baac-0208ff8642ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:19.718093 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1330/2687] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:19 2022] DELETE /identity/v3/projects/ddbb72d18d3e46f19f18c88b1382e36c => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:19.723073 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935a27a-9599-4b17-a15c-a48ab75cab9a None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:19.723552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935a27a-9599-4b17-a15c-a48ab75cab9a None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:19.723662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935a27a-9599-4b17-a15c-a48ab75cab9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:19.744546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-5935a27a-9599-4b17-a15c-a48ab75cab9a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:19.745767 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-5935a27a-9599-4b17-a15c-a48ab75cab9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:19.790821 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:19.809161 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1358/2688] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:19.818287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcce26-e2e1-4ba6-8678-322d06c4e223 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:19.819726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcce26-e2e1-4ba6-8678-322d06c4e223 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:19.819726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcce26-e2e1-4ba6-8678-322d06c4e223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:19.840894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-9bdcce26-e2e1-4ba6-8678-322d06c4e223 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:19.843270 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9bdcce26-e2e1-4ba6-8678-322d06c4e223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:19.851363 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1331/2689] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:19.855832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935a27a-9599-4b17-a15c-a48ab75cab9a None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:19.866596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935a27a-9599-4b17-a15c-a48ab75cab9a None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:19.871302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b68b9-f52f-4d1b-a650-f8d13d5aac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:19.871906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b68b9-f52f-4d1b-a650-f8d13d5aac58 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:19.872217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b68b9-f52f-4d1b-a650-f8d13d5aac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:19.872496 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b68b9-f52f-4d1b-a650-f8d13d5aac58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:19.873157 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b68b9-f52f-4d1b-a650-f8d13d5aac58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:01:19.873999 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b68b9-f52f-4d1b-a650-f8d13d5aac58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:19.875682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b68b9-f52f-4d1b-a650-f8d13d5aac58 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:19.887810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1359/2690] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:01:19.893618 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcce26-e2e1-4ba6-8678-322d06c4e223 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:19.903172 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcce26-e2e1-4ba6-8678-322d06c4e223 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:19.908895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d9822-2998-4560-a9f5-a82cee73960a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:19.909512 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d9822-2998-4560-a9f5-a82cee73960a admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:19.909796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d9822-2998-4560-a9f5-a82cee73960a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:19.910071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d9822-2998-4560-a9f5-a82cee73960a admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:19.910743 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d9822-2998-4560-a9f5-a82cee73960a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:19.912666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d9822-2998-4560-a9f5-a82cee73960a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:20.052057 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1332/2691] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:19 2022] POST /identity/v3/projects => generated 365 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:20.053389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b68b9-f52f-4d1b-a650-f8d13d5aac58 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:20.064748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b68b9-f52f-4d1b-a650-f8d13d5aac58 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:20.124362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855387e-fc17-4743-b28b-37e63b3249bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:20.124976 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855387e-fc17-4743-b28b-37e63b3249bc admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:20.125283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855387e-fc17-4743-b28b-37e63b3249bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:20.125551 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855387e-fc17-4743-b28b-37e63b3249bc admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:20.126199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855387e-fc17-4743-b28b-37e63b3249bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:20.128494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855387e-fc17-4743-b28b-37e63b3249bc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:20.217757 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1360/2692] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:20 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:20.222090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d9822-2998-4560-a9f5-a82cee73960a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:20.232789 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d9822-2998-4560-a9f5-a82cee73960a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:20.237396 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994c424-c650-4523-9723-fd2fa48aea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:20.237998 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994c424-c650-4523-9723-fd2fa48aea77 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:20.238283 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994c424-c650-4523-9723-fd2fa48aea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:20.238585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994c424-c650-4523-9723-fd2fa48aea77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:20.239235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994c424-c650-4523-9723-fd2fa48aea77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:20.240849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994c424-c650-4523-9723-fd2fa48aea77 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:20.253557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1333/2693] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:20.256446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855387e-fc17-4743-b28b-37e63b3249bc admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:20.267975 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855387e-fc17-4743-b28b-37e63b3249bc admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:20.272907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a4e26-4307-4b78-b237-438f7f525642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:20.273519 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a4e26-4307-4b78-b237-438f7f525642 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:20.273801 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a4e26-4307-4b78-b237-438f7f525642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:20.274081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a4e26-4307-4b78-b237-438f7f525642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:20.274765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a4e26-4307-4b78-b237-438f7f525642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:20.276385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a4e26-4307-4b78-b237-438f7f525642 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:20.288320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1361/2694] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:20.292757 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994c424-c650-4523-9723-fd2fa48aea77 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:20.304507 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994c424-c650-4523-9723-fd2fa48aea77 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:20.310333 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5adf2-5f3b-414d-9f11-1e67689323eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:20.312812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5adf2-5f3b-414d-9f11-1e67689323eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:20.312812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5adf2-5f3b-414d-9f11-1e67689323eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:20.312812 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5adf2-5f3b-414d-9f11-1e67689323eb admin admin] PATH_INFO: `/v3/projects/1eefabc148c14436a7d450c46f8a1eba/users/73d8073db8054543adac323827ea2377/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:20.331107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5adf2-5f3b-414d-9f11-1e67689323eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eefabc148c14436a7d450c46f8a1eba, user_id=73d8073db8054543adac323827ea2377, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:20.335688 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5adf2-5f3b-414d-9f11-1e67689323eb admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:20.424888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1334/2695] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:20 2022] PUT /identity/v3/projects/1eefabc148c14436a7d450c46f8a1eba/users/73d8073db8054543adac323827ea2377/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:20.443114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8925692-7ea0-4f52-9a2d-582b03aaffa2 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:20.443402 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8925692-7ea0-4f52-9a2d-582b03aaffa2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:20.443683 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8925692-7ea0-4f52-9a2d-582b03aaffa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:20.473180 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-e8925692-7ea0-4f52-9a2d-582b03aaffa2 None None] MFA Rules not processed for user `73d8073db8054543adac323827ea2377`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:20.474432 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-e8925692-7ea0-4f52-9a2d-582b03aaffa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:20.542276 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1362/2696] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:20.547490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5adf2-5f3b-414d-9f11-1e67689323eb admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:20.613355 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5adf2-5f3b-414d-9f11-1e67689323eb admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:20.618148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb388e1-90ea-40ed-9e52-716a3d2d4c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:20.618798 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb388e1-90ea-40ed-9e52-716a3d2d4c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:20.619080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb388e1-90ea-40ed-9e52-716a3d2d4c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:20.619360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb388e1-90ea-40ed-9e52-716a3d2d4c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:20.619981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb388e1-90ea-40ed-9e52-716a3d2d4c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:20.621931 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb388e1-90ea-40ed-9e52-716a3d2d4c00 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:20.734986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1335/2697] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:20 2022] POST /identity/v3/projects => generated 365 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:20.740806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8925692-7ea0-4f52-9a2d-582b03aaffa2 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:20.752970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8925692-7ea0-4f52-9a2d-582b03aaffa2 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:20.796382 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:20.813985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270e1eb-4d87-4429-ac7f-ff4eeb746548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:20.814644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270e1eb-4d87-4429-ac7f-ff4eeb746548 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:20.814930 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270e1eb-4d87-4429-ac7f-ff4eeb746548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:20.815206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270e1eb-4d87-4429-ac7f-ff4eeb746548 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:20.815841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270e1eb-4d87-4429-ac7f-ff4eeb746548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:20.818135 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270e1eb-4d87-4429-ac7f-ff4eeb746548 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:20.932996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1363/2698] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:20 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:20.938393 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb388e1-90ea-40ed-9e52-716a3d2d4c00 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:20.948926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb388e1-90ea-40ed-9e52-716a3d2d4c00 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:20.954070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80dea4-c166-46a5-84fe-4678e9eec0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:20.960780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80dea4-c166-46a5-84fe-4678e9eec0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:20.960780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80dea4-c166-46a5-84fe-4678e9eec0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:20.960780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80dea4-c166-46a5-84fe-4678e9eec0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:20.960780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80dea4-c166-46a5-84fe-4678e9eec0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:20.960780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80dea4-c166-46a5-84fe-4678e9eec0f5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:20.969137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1336/2699] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:20.984820 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270e1eb-4d87-4429-ac7f-ff4eeb746548 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:20.997164 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270e1eb-4d87-4429-ac7f-ff4eeb746548 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:21.010218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bb49c-dcf0-496e-8b2b-26681a0c18c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:21.011091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb49c-dcf0-496e-8b2b-26681a0c18c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:21.011442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb49c-dcf0-496e-8b2b-26681a0c18c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:21.015489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb49c-dcf0-496e-8b2b-26681a0c18c0 admin admin] PATH_INFO: `/v3/projects/82bd040f3acd4f629cc153b2d48caff5/users/afedd9e8bf854315b9084b38d8006a97/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:21.029069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bb49c-dcf0-496e-8b2b-26681a0c18c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82bd040f3acd4f629cc153b2d48caff5, user_id=afedd9e8bf854315b9084b38d8006a97, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:21.032662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bb49c-dcf0-496e-8b2b-26681a0c18c0 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:21.159989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1364/2700] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:20 2022] PUT /identity/v3/projects/82bd040f3acd4f629cc153b2d48caff5/users/afedd9e8bf854315b9084b38d8006a97/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:21.164327 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80dea4-c166-46a5-84fe-4678e9eec0f5 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:21.176160 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80dea4-c166-46a5-84fe-4678e9eec0f5 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:21.197219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:21.245859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeee97-b00e-429f-a38d-c66ea7cadaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:21.245859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeee97-b00e-429f-a38d-c66ea7cadaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:21.245859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeee97-b00e-429f-a38d-c66ea7cadaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:21.245859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeee97-b00e-429f-a38d-c66ea7cadaa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:21.246399 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eeee97-b00e-429f-a38d-c66ea7cadaa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:21.247680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eeee97-b00e-429f-a38d-c66ea7cadaa0 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:21.257997 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1337/2701] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:21.264328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bb49c-dcf0-496e-8b2b-26681a0c18c0 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:21.277372 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bb49c-dcf0-496e-8b2b-26681a0c18c0 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:21.283792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73559d-a585-41f3-86fd-6e5ac33ee1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:21.284703 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73559d-a585-41f3-86fd-6e5ac33ee1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:21.285055 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73559d-a585-41f3-86fd-6e5ac33ee1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:21.285512 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73559d-a585-41f3-86fd-6e5ac33ee1a9 admin admin] PATH_INFO: `/v3/projects/82bd040f3acd4f629cc153b2d48caff5/users/afedd9e8bf854315b9084b38d8006a97/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:21.287611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73559d-a585-41f3-86fd-6e5ac33ee1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82bd040f3acd4f629cc153b2d48caff5, user_id=afedd9e8bf854315b9084b38d8006a97, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:21.290258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73559d-a585-41f3-86fd-6e5ac33ee1a9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:21.306802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1365/2702] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:21 2022] PUT /identity/v3/projects/82bd040f3acd4f629cc153b2d48caff5/users/afedd9e8bf854315b9084b38d8006a97/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:21.318378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fffb3-a653-4fe2-a982-866ace0be9c9 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:21.320443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fffb3-a653-4fe2-a982-866ace0be9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:21.321443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475fffb3-a653-4fe2-a982-866ace0be9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:21.351585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-475fffb3-a653-4fe2-a982-866ace0be9c9 None None] MFA Rules not processed for user `afedd9e8bf854315b9084b38d8006a97`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:21.352844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-475fffb3-a653-4fe2-a982-866ace0be9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:21.425749 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1338/2703] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:21.437859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73559d-a585-41f3-86fd-6e5ac33ee1a9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:21.439773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bc73559d-a585-41f3-86fd-6e5ac33ee1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:21.499546 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73559d-a585-41f3-86fd-6e5ac33ee1a9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:21.503885 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd913e-d3cb-469b-b9a5-31eb1b22f239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:21.504491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd913e-d3cb-469b-b9a5-31eb1b22f239 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:21.504774 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd913e-d3cb-469b-b9a5-31eb1b22f239 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:21.505052 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd913e-d3cb-469b-b9a5-31eb1b22f239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:21.515321 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd913e-d3cb-469b-b9a5-31eb1b22f239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:21.516826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd913e-d3cb-469b-b9a5-31eb1b22f239 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:21.526718 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1366/2704] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:22.797735 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475fffb3-a653-4fe2-a982-866ace0be9c9 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:22.872405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475fffb3-a653-4fe2-a982-866ace0be9c9 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:22.877918 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:22.879244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:22.879244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:22.880785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] PATH_INFO: `/v3/users/73d8073db8054543adac323827ea2377` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:22.880785 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d8073db8054543adac323827ea2377)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:22.882390 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:23.091906 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d8073db8054543adac323827ea2377'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:23.094222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:23.178925 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d8073db8054543adac323827ea2377'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:23.193990 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d8073db8054543adac323827ea2377'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:23.201227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d8073db8054543adac323827ea2377'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:23.201978 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d8073db8054543adac323827ea2377'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:23.208595 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1339/2705] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:22 2022] DELETE /identity/v3/users/73d8073db8054543adac323827ea2377 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:23.214305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98948d-df51-42f3-a362-c8f21f72beff None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:23.214305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98948d-df51-42f3-a362-c8f21f72beff None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:23.214305 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98948d-df51-42f3-a362-c8f21f72beff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:23.238610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-3e98948d-df51-42f3-a362-c8f21f72beff None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:23.240615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3e98948d-df51-42f3-a362-c8f21f72beff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:23.308854 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1367/2706] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:23.319873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:23.321450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:23.389810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727a35d-323f-431e-b067-fc4894fb64c5 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:23.393394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ca282-024b-41a8-941f-5c4825825cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:23.394667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ca282-024b-41a8-941f-5c4825825cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:23.394947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ca282-024b-41a8-941f-5c4825825cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:23.395223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ca282-024b-41a8-941f-5c4825825cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:23.405313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ca282-024b-41a8-941f-5c4825825cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:23.406794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ca282-024b-41a8-941f-5c4825825cb1 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:23.416736 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1340/2707] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:23.923501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98948d-df51-42f3-a362-c8f21f72beff None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:23.924919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-3e98948d-df51-42f3-a362-c8f21f72beff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:23.939861 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98948d-df51-42f3-a362-c8f21f72beff None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:23.945283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:23.945958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:23.946228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:23.946579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] PATH_INFO: `/v3/projects/1eefabc148c14436a7d450c46f8a1eba` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:23.947709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eefabc148c14436a7d450c46f8a1eba)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:23.949698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:24.053745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:24.065827 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eefabc148c14436a7d450c46f8a1eba', 'request_id': 'req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.071374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eefabc148c14436a7d450c46f8a1eba', 'request_id': 'req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.130853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eefabc148c14436a7d450c46f8a1eba', 'request_id': 'req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.142776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] The token cache is being invalidate because project 1eefabc148c14436a7d450c46f8a1eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:24.143264 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.144380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.145924 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1368/2708] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:23 2022] DELETE /identity/v3/projects/1eefabc148c14436a7d450c46f8a1eba => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:24.150023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ca282-024b-41a8-941f-5c4825825cb1 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:24.160847 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ca282-024b-41a8-941f-5c4825825cb1 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:24.217324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:24.217883 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:24.218228 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:24.218531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] PATH_INFO: `/v3/users/afedd9e8bf854315b9084b38d8006a97` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:24.219680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afedd9e8bf854315b9084b38d8006a97)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:24.221599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:24.461740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afedd9e8bf854315b9084b38d8006a97'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.549702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afedd9e8bf854315b9084b38d8006a97'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.563252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:24.564726 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afedd9e8bf854315b9084b38d8006a97'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.572407 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afedd9e8bf854315b9084b38d8006a97'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.573223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1f536561-d469-4632-8c40-b71220df23a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afedd9e8bf854315b9084b38d8006a97'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:24.580038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1341/2709] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:24 2022] DELETE /identity/v3/users/afedd9e8bf854315b9084b38d8006a97 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:25.035680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:25.038045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:25.054457 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461cc9c-6566-4a3a-bd41-4b79b6f59ae5 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:25.118964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:25.119599 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:25.119879 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:25.120151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] PATH_INFO: `/v3/projects/82bd040f3acd4f629cc153b2d48caff5` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:25.121270 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=82bd040f3acd4f629cc153b2d48caff5)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:25.123803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:25.249622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82bd040f3acd4f629cc153b2d48caff5', 'request_id': 'req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:25.255283 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82bd040f3acd4f629cc153b2d48caff5', 'request_id': 'req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:25.355627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82bd040f3acd4f629cc153b2d48caff5', 'request_id': 'req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:25.367732 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] The token cache is being invalidate because project 82bd040f3acd4f629cc153b2d48caff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:25.368022 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:25.368650 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-5c2b70f0-0f4f-4758-9186-fd3aa32e3c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:25.370136 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1369/2710] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:25 2022] DELETE /identity/v3/projects/82bd040f3acd4f629cc153b2d48caff5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:25.375741 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434cd11-5846-469d-89df-a23ada64873b None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:25.376227 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434cd11-5846-469d-89df-a23ada64873b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:25.376454 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434cd11-5846-469d-89df-a23ada64873b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:25.396035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-3434cd11-5846-469d-89df-a23ada64873b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:25.397707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3434cd11-5846-469d-89df-a23ada64873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:25.456041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1342/2711] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:25.463833 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1326fb-689e-48ad-ba5f-2db16a026cba None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:25.472254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1326fb-689e-48ad-ba5f-2db16a026cba None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:25.472254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1326fb-689e-48ad-ba5f-2db16a026cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:25.486326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-4a1326fb-689e-48ad-ba5f-2db16a026cba None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:25.487596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-4a1326fb-689e-48ad-ba5f-2db16a026cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:25.495819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1370/2712] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:25.500372 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434cd11-5846-469d-89df-a23ada64873b None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:25.510623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434cd11-5846-469d-89df-a23ada64873b None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:25.515379 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f78f-0f45-4c39-99b1-4ad79bb5568d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:25.515981 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f78f-0f45-4c39-99b1-4ad79bb5568d admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:25.516262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f78f-0f45-4c39-99b1-4ad79bb5568d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:25.516533 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f78f-0f45-4c39-99b1-4ad79bb5568d admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:25.517180 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9f78f-0f45-4c39-99b1-4ad79bb5568d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:01:25.517485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9f78f-0f45-4c39-99b1-4ad79bb5568d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:25.519114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9f78f-0f45-4c39-99b1-4ad79bb5568d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:25.532273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1343/2713] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:01:25.536808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1326fb-689e-48ad-ba5f-2db16a026cba None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:25.547405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1326fb-689e-48ad-ba5f-2db16a026cba None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:25.552346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b27ce-ac3e-48e4-83bb-cc1f641bfa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:25.552962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b27ce-ac3e-48e4-83bb-cc1f641bfa30 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:25.553277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b27ce-ac3e-48e4-83bb-cc1f641bfa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:25.553556 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b27ce-ac3e-48e4-83bb-cc1f641bfa30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:25.554206 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b27ce-ac3e-48e4-83bb-cc1f641bfa30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:25.556263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b27ce-ac3e-48e4-83bb-cc1f641bfa30 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:25.689118 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1371/2714] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:25 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:25.693319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f78f-0f45-4c39-99b1-4ad79bb5568d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:25.703495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f78f-0f45-4c39-99b1-4ad79bb5568d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:25.758797 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292d33f-7eee-4753-9343-749b7fd9a81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:25.759397 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292d33f-7eee-4753-9343-749b7fd9a81a admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:25.759678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292d33f-7eee-4753-9343-749b7fd9a81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:25.759952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292d33f-7eee-4753-9343-749b7fd9a81a admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:25.760572 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292d33f-7eee-4753-9343-749b7fd9a81a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:25.762488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292d33f-7eee-4753-9343-749b7fd9a81a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:25.861439 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1344/2715] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:25 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:01:25.864254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b27ce-ac3e-48e4-83bb-cc1f641bfa30 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:25.873438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b27ce-ac3e-48e4-83bb-cc1f641bfa30 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:25.878684 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048eb20-fe68-4b12-8bb1-b369b0a39f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:25.879358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048eb20-fe68-4b12-8bb1-b369b0a39f36 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:25.879776 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048eb20-fe68-4b12-8bb1-b369b0a39f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:25.880180 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048eb20-fe68-4b12-8bb1-b369b0a39f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:25.880836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048eb20-fe68-4b12-8bb1-b369b0a39f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:25.882475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048eb20-fe68-4b12-8bb1-b369b0a39f36 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:25.896736 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1372/2716] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:25.898202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292d33f-7eee-4753-9343-749b7fd9a81a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:25.912511 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292d33f-7eee-4753-9343-749b7fd9a81a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:25.918942 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c848bb-3318-4b0e-b570-3b074d4af291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:25.919876 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c848bb-3318-4b0e-b570-3b074d4af291 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:25.920274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c848bb-3318-4b0e-b570-3b074d4af291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:25.920651 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c848bb-3318-4b0e-b570-3b074d4af291 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:25.921446 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c848bb-3318-4b0e-b570-3b074d4af291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:25.923148 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c848bb-3318-4b0e-b570-3b074d4af291 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:25.941045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1345/2717] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:25.943133 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048eb20-fe68-4b12-8bb1-b369b0a39f36 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:25.954928 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048eb20-fe68-4b12-8bb1-b369b0a39f36 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:25.959978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923ec87-15e1-4e70-9a06-89e7fa41fe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:25.960714 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923ec87-15e1-4e70-9a06-89e7fa41fe97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:25.961066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923ec87-15e1-4e70-9a06-89e7fa41fe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:25.961409 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923ec87-15e1-4e70-9a06-89e7fa41fe97 admin admin] PATH_INFO: `/v3/projects/4c9ece851f634c6f868b9257b5d5442f/users/12f230cf3023468db08a4521c2278aaf/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:25.971121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:25.978621 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923ec87-15e1-4e70-9a06-89e7fa41fe97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9ece851f634c6f868b9257b5d5442f, user_id=12f230cf3023468db08a4521c2278aaf, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:25.981507 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923ec87-15e1-4e70-9a06-89e7fa41fe97 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:26.070414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1373/2718] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:25 2022] PUT /identity/v3/projects/4c9ece851f634c6f868b9257b5d5442f/users/12f230cf3023468db08a4521c2278aaf/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:26.081471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a347-8344-4b11-95e9-1161232cd051 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:26.081755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a347-8344-4b11-95e9-1161232cd051 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:26.082024 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a347-8344-4b11-95e9-1161232cd051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:26.110774 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d973a347-8344-4b11-95e9-1161232cd051 None None] MFA Rules not processed for user `12f230cf3023468db08a4521c2278aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:26.111975 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d973a347-8344-4b11-95e9-1161232cd051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:26.161193 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:26.183971 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1346/2719] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:26.189598 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923ec87-15e1-4e70-9a06-89e7fa41fe97 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:26.252841 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923ec87-15e1-4e70-9a06-89e7fa41fe97 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:26.257416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28024c-5e99-46ee-9ff7-cc7cdcaab138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:26.258021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28024c-5e99-46ee-9ff7-cc7cdcaab138 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:26.258304 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28024c-5e99-46ee-9ff7-cc7cdcaab138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:26.258600 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28024c-5e99-46ee-9ff7-cc7cdcaab138 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:26.259229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28024c-5e99-46ee-9ff7-cc7cdcaab138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:26.261217 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28024c-5e99-46ee-9ff7-cc7cdcaab138 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:26.344167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1374/2720] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:26 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:26.348513 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973a347-8344-4b11-95e9-1161232cd051 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:26.359742 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973a347-8344-4b11-95e9-1161232cd051 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:26.422690 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37881ad9-686c-402a-868e-057cbbb5cd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:26.423378 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37881ad9-686c-402a-868e-057cbbb5cd32 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:26.423569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37881ad9-686c-402a-868e-057cbbb5cd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:26.423846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37881ad9-686c-402a-868e-057cbbb5cd32 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:26.424470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37881ad9-686c-402a-868e-057cbbb5cd32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:26.426434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37881ad9-686c-402a-868e-057cbbb5cd32 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:26.493578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1347/2721] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:26 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:26.497706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28024c-5e99-46ee-9ff7-cc7cdcaab138 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:26.509411 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28024c-5e99-46ee-9ff7-cc7cdcaab138 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:26.514166 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f40521-ffc5-4bf3-a0b1-6d7ae59ad800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:26.514877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f40521-ffc5-4bf3-a0b1-6d7ae59ad800 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:26.515160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f40521-ffc5-4bf3-a0b1-6d7ae59ad800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:26.515441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f40521-ffc5-4bf3-a0b1-6d7ae59ad800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:26.516122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f40521-ffc5-4bf3-a0b1-6d7ae59ad800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:26.517773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f40521-ffc5-4bf3-a0b1-6d7ae59ad800 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:26.528464 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1375/2722] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:26.532868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37881ad9-686c-402a-868e-057cbbb5cd32 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:26.542926 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37881ad9-686c-402a-868e-057cbbb5cd32 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:26.548339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb3248-eb20-4e0c-a605-2718b0591ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:26.548986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb3248-eb20-4e0c-a605-2718b0591ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:26.549267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb3248-eb20-4e0c-a605-2718b0591ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:26.549546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb3248-eb20-4e0c-a605-2718b0591ab7 admin admin] PATH_INFO: `/v3/projects/1c96990260e446a399f0e3a845ed60a6/users/16a3380962fe451ea2bcbe558407ec5c/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:26.565125 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb3248-eb20-4e0c-a605-2718b0591ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c96990260e446a399f0e3a845ed60a6, user_id=16a3380962fe451ea2bcbe558407ec5c, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:26.567890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb3248-eb20-4e0c-a605-2718b0591ab7 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:26.681718 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1348/2723] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:26 2022] PUT /identity/v3/projects/1c96990260e446a399f0e3a845ed60a6/users/16a3380962fe451ea2bcbe558407ec5c/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:26.686596 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f40521-ffc5-4bf3-a0b1-6d7ae59ad800 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:26.699023 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f40521-ffc5-4bf3-a0b1-6d7ae59ad800 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:26.776275 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141df9-c686-44dd-8ef2-5687fdd13a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:26.776896 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141df9-c686-44dd-8ef2-5687fdd13a8f admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:26.777182 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141df9-c686-44dd-8ef2-5687fdd13a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:26.777461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141df9-c686-44dd-8ef2-5687fdd13a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:26.778117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141df9-c686-44dd-8ef2-5687fdd13a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:26.779829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141df9-c686-44dd-8ef2-5687fdd13a8f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:26.790061 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1376/2724] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:26.795437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb3248-eb20-4e0c-a605-2718b0591ab7 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:26.805752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb3248-eb20-4e0c-a605-2718b0591ab7 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:26.811219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6f5d8-2a3b-4660-a375-1590c1033ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:26.811979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6f5d8-2a3b-4660-a375-1590c1033ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:26.812265 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6f5d8-2a3b-4660-a375-1590c1033ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:26.812546 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6f5d8-2a3b-4660-a375-1590c1033ad4 admin admin] PATH_INFO: `/v3/projects/1c96990260e446a399f0e3a845ed60a6/users/16a3380962fe451ea2bcbe558407ec5c/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:26.814492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6f5d8-2a3b-4660-a375-1590c1033ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c96990260e446a399f0e3a845ed60a6, user_id=16a3380962fe451ea2bcbe558407ec5c, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:26.817164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6f5d8-2a3b-4660-a375-1590c1033ad4 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:26.827708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1349/2725] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:26 2022] PUT /identity/v3/projects/1c96990260e446a399f0e3a845ed60a6/users/16a3380962fe451ea2bcbe558407ec5c/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:26.838711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4c67e-918c-4b6a-b196-65a28c0881a2 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:26.839005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4c67e-918c-4b6a-b196-65a28c0881a2 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:26.839289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4c67e-918c-4b6a-b196-65a28c0881a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:26.867256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-52e4c67e-918c-4b6a-b196-65a28c0881a2 None None] MFA Rules not processed for user `16a3380962fe451ea2bcbe558407ec5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:26.868497 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-52e4c67e-918c-4b6a-b196-65a28c0881a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:26.914492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:26.937906 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1377/2726] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:26.948488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6f5d8-2a3b-4660-a375-1590c1033ad4 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:26.950187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d1c6f5d8-2a3b-4660-a375-1590c1033ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:27.013110 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6f5d8-2a3b-4660-a375-1590c1033ad4 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:27.018764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c65eca-d034-48aa-991d-76f32a6945b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:27.019389 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c65eca-d034-48aa-991d-76f32a6945b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:27.019667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c65eca-d034-48aa-991d-76f32a6945b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:27.019937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c65eca-d034-48aa-991d-76f32a6945b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:27.036167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c65eca-d034-48aa-991d-76f32a6945b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:27.037765 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c65eca-d034-48aa-991d-76f32a6945b9 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:27.051602 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1350/2727] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:27.971479 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4c67e-918c-4b6a-b196-65a28c0881a2 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:27.973504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-52e4c67e-918c-4b6a-b196-65a28c0881a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:28.057122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4c67e-918c-4b6a-b196-65a28c0881a2 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:28.068098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:28.068098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:28.068098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:28.068098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] PATH_INFO: `/v3/users/2206cb5005354ab4a716efb525c633e7` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:28.068673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2206cb5005354ab4a716efb525c633e7)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:28.068673 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:28.322406 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2206cb5005354ab4a716efb525c633e7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:28.323121 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2206cb5005354ab4a716efb525c633e7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:28.400835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2206cb5005354ab4a716efb525c633e7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:28.408887 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2206cb5005354ab4a716efb525c633e7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:28.421249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-1519b039-5102-46da-9da9-dcdd5551b7bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2206cb5005354ab4a716efb525c633e7'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:28.430285 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1378/2728] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:27 2022] DELETE /identity/v3/users/2206cb5005354ab4a716efb525c633e7 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:28.742097 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c65eca-d034-48aa-991d-76f32a6945b9 service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:28.758042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c65eca-d034-48aa-991d-76f32a6945b9 service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:28.797433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:28.822712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:28.823297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:28.823583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:28.823859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] PATH_INFO: `/v3/projects/6e3d57c8abbd49e69c0caad57f7f9815` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:28.824992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3d57c8abbd49e69c0caad57f7f9815)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:28.827623 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:28.937116 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3d57c8abbd49e69c0caad57f7f9815', 'request_id': 'req-ae193725-6ac5-46ec-99e1-8ba6d94bd667'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:29.013006 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3d57c8abbd49e69c0caad57f7f9815', 'request_id': 'req-ae193725-6ac5-46ec-99e1-8ba6d94bd667'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:29.028489 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3d57c8abbd49e69c0caad57f7f9815', 'request_id': 'req-ae193725-6ac5-46ec-99e1-8ba6d94bd667'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:29.040868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] The token cache is being invalidate because project 6e3d57c8abbd49e69c0caad57f7f9815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:29.040868 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:29.052381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ae193725-6ac5-46ec-99e1-8ba6d94bd667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:29.052381 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1351/2729] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:28 2022] DELETE /identity/v3/projects/6e3d57c8abbd49e69c0caad57f7f9815 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:29.055629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226017dd-93f0-4fe3-976c-836938fd2bf6 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:29.056105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226017dd-93f0-4fe3-976c-836938fd2bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:29.056203 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226017dd-93f0-4fe3-976c-836938fd2bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:29.077064 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-226017dd-93f0-4fe3-976c-836938fd2bf6 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:29.078282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-226017dd-93f0-4fe3-976c-836938fd2bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:29.143400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1379/2730] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:29.150620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037a22a-6180-463c-8b23-a0412e8b51c3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:29.150912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037a22a-6180-463c-8b23-a0412e8b51c3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:29.151189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037a22a-6180-463c-8b23-a0412e8b51c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:29.173042 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-1037a22a-6180-463c-8b23-a0412e8b51c3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:29.174287 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1037a22a-6180-463c-8b23-a0412e8b51c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:29.182240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1352/2731] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:29.186742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226017dd-93f0-4fe3-976c-836938fd2bf6 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:29.191675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:29.197049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226017dd-93f0-4fe3-976c-836938fd2bf6 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:29.203081 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8071669-0fdc-492f-b6c7-605de7e945a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:29.203707 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8071669-0fdc-492f-b6c7-605de7e945a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:29.203989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8071669-0fdc-492f-b6c7-605de7e945a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:29.204263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8071669-0fdc-492f-b6c7-605de7e945a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:29.205045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8071669-0fdc-492f-b6c7-605de7e945a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:01:29.205352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8071669-0fdc-492f-b6c7-605de7e945a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:29.207119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8071669-0fdc-492f-b6c7-605de7e945a7 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:29.219746 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1380/2732] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:01:29.224069 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037a22a-6180-463c-8b23-a0412e8b51c3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:29.234370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037a22a-6180-463c-8b23-a0412e8b51c3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:29.239012 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2f89a-3a09-4fac-a66c-f10280ba3375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:29.239609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2f89a-3a09-4fac-a66c-f10280ba3375 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:29.239888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2f89a-3a09-4fac-a66c-f10280ba3375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:29.240161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2f89a-3a09-4fac-a66c-f10280ba3375 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:29.240793 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2f89a-3a09-4fac-a66c-f10280ba3375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:29.242866 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2f89a-3a09-4fac-a66c-f10280ba3375 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:29.384582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1353/2733] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:29 2022] POST /identity/v3/projects => generated 391 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:29.388873 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8071669-0fdc-492f-b6c7-605de7e945a7 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:29.400576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8071669-0fdc-492f-b6c7-605de7e945a7 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:29.484216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213a3f7-c7fd-4046-8575-80328d026977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:29.484748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213a3f7-c7fd-4046-8575-80328d026977 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:29.485033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213a3f7-c7fd-4046-8575-80328d026977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:29.485327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213a3f7-c7fd-4046-8575-80328d026977 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:29.486033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213a3f7-c7fd-4046-8575-80328d026977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:29.490658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213a3f7-c7fd-4046-8575-80328d026977 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:29.694050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1381/2734] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:29 2022] POST /identity/v3/users => generated 296 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:01:29.697433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2f89a-3a09-4fac-a66c-f10280ba3375 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:29.715198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2f89a-3a09-4fac-a66c-f10280ba3375 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:29.722947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd82572-9de3-489a-a57c-5b94e97059a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:29.723899 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd82572-9de3-489a-a57c-5b94e97059a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:29.724334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd82572-9de3-489a-a57c-5b94e97059a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:29.725088 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd82572-9de3-489a-a57c-5b94e97059a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:29.725917 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd82572-9de3-489a-a57c-5b94e97059a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:29.728629 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd82572-9de3-489a-a57c-5b94e97059a4 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:29.746419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1354/2735] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:29.750745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213a3f7-c7fd-4046-8575-80328d026977 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:29.768286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213a3f7-c7fd-4046-8575-80328d026977 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:29.773403 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca4c1f-bdcf-4368-9992-16688b8ee786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:29.774040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca4c1f-bdcf-4368-9992-16688b8ee786 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:29.774412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca4c1f-bdcf-4368-9992-16688b8ee786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:29.774627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca4c1f-bdcf-4368-9992-16688b8ee786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:29.775282 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca4c1f-bdcf-4368-9992-16688b8ee786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:29.776900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca4c1f-bdcf-4368-9992-16688b8ee786 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:29.794583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1382/2736] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:29.796208 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd82572-9de3-489a-a57c-5b94e97059a4 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:29.806810 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd82572-9de3-489a-a57c-5b94e97059a4 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:29.812475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bf249-28a8-4a20-a4d9-a7f8fa90bf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:29.813194 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bf249-28a8-4a20-a4d9-a7f8fa90bf62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:29.813604 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bf249-28a8-4a20-a4d9-a7f8fa90bf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:29.813892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bf249-28a8-4a20-a4d9-a7f8fa90bf62 admin admin] PATH_INFO: `/v3/projects/9d25382b4a8e404bbc5a823cdf6f624e/users/442f28e73ed9496a8c762093f31a9594/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:29.831317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bf249-28a8-4a20-a4d9-a7f8fa90bf62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d25382b4a8e404bbc5a823cdf6f624e, user_id=442f28e73ed9496a8c762093f31a9594, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:29.834455 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bf249-28a8-4a20-a4d9-a7f8fa90bf62 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:29.978153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1355/2737] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:29 2022] PUT /identity/v3/projects/9d25382b4a8e404bbc5a823cdf6f624e/users/442f28e73ed9496a8c762093f31a9594/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:29.988517 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4d205-8912-4b54-a1cc-ea00e9cb7087 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:29.988810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4d205-8912-4b54-a1cc-ea00e9cb7087 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:29.989125 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4d205-8912-4b54-a1cc-ea00e9cb7087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:30.018143 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d7b4d205-8912-4b54-a1cc-ea00e9cb7087 None None] MFA Rules not processed for user `442f28e73ed9496a8c762093f31a9594`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:30.019419 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d7b4d205-8912-4b54-a1cc-ea00e9cb7087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:30.091503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1383/2738] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:30.102775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bf249-28a8-4a20-a4d9-a7f8fa90bf62 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:30.104658 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-681bf249-28a8-4a20-a4d9-a7f8fa90bf62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:30.162964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4d205-8912-4b54-a1cc-ea00e9cb7087 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:30.164326 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d7b4d205-8912-4b54-a1cc-ea00e9cb7087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:30.170527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bf249-28a8-4a20-a4d9-a7f8fa90bf62 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:30.174554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5ad6d-fc34-4b4b-9695-b98d62c6ed86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:30.175171 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5ad6d-fc34-4b4b-9695-b98d62c6ed86 service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:30.175453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5ad6d-fc34-4b4b-9695-b98d62c6ed86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:30.175729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5ad6d-fc34-4b4b-9695-b98d62c6ed86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:30.186368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5ad6d-fc34-4b4b-9695-b98d62c6ed86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:30.188260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5ad6d-fc34-4b4b-9695-b98d62c6ed86 service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:30.197648 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1356/2739] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:30.230611 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:30.235638 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4d205-8912-4b54-a1cc-ea00e9cb7087 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:30.240992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:30.241501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:30.241788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:30.242068 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] PATH_INFO: `/v3/users/12f230cf3023468db08a4521c2278aaf` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:30.243236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f230cf3023468db08a4521c2278aaf)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:30.245298 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:30.422670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f230cf3023468db08a4521c2278aaf'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:30.423484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f230cf3023468db08a4521c2278aaf'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:30.475032 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f230cf3023468db08a4521c2278aaf'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:30.483107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f230cf3023468db08a4521c2278aaf'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:30.495501 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f230cf3023468db08a4521c2278aaf'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:30.502759 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1384/2740] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:30 2022] DELETE /identity/v3/users/12f230cf3023468db08a4521c2278aaf => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:01:30.507040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f942c-1fb6-4c86-a7e6-89606ec3e5d3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:30.507325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f942c-1fb6-4c86-a7e6-89606ec3e5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:30.507960 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f942c-1fb6-4c86-a7e6-89606ec3e5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:30.523223 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:30.529844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-9b1f942c-1fb6-4c86-a7e6-89606ec3e5d3 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:30.531300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9b1f942c-1fb6-4c86-a7e6-89606ec3e5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:30.591813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1357/2741] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:30.602407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:30.603882 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:30.664038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4837fd-7877-458b-b52c-68722c5e558d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:30.667826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7822405-3898-4b01-a050-0468a49f4247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:30.668436 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7822405-3898-4b01-a050-0468a49f4247 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:30.668720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7822405-3898-4b01-a050-0468a49f4247 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:30.668998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7822405-3898-4b01-a050-0468a49f4247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:30.679416 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7822405-3898-4b01-a050-0468a49f4247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:30.680800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7822405-3898-4b01-a050-0468a49f4247 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:30.690522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1385/2742] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:01:31.164198 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f942c-1fb6-4c86-a7e6-89606ec3e5d3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:31.165820 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9b1f942c-1fb6-4c86-a7e6-89606ec3e5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:31.184828 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f942c-1fb6-4c86-a7e6-89606ec3e5d3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:31.189131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:31.189795 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:31.190080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:31.190416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] PATH_INFO: `/v3/projects/4c9ece851f634c6f868b9257b5d5442f` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:31.191606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9ece851f634c6f868b9257b5d5442f)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:31.193596 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:31.311161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9ece851f634c6f868b9257b5d5442f', 'request_id': 'req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.378408 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9ece851f634c6f868b9257b5d5442f', 'request_id': 'req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.385215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9ece851f634c6f868b9257b5d5442f', 'request_id': 'req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.396336 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] The token cache is being invalidate because project 4c9ece851f634c6f868b9257b5d5442f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:31.396625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.397345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.398944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1358/2743] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:31 2022] DELETE /identity/v3/projects/4c9ece851f634c6f868b9257b5d5442f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:31.403215 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7822405-3898-4b01-a050-0468a49f4247 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:31.412891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7822405-3898-4b01-a050-0468a49f4247 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:31.479563 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:31.480132 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:31.480404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:31.480676 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] PATH_INFO: `/v3/users/16a3380962fe451ea2bcbe558407ec5c` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:31.481829 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a3380962fe451ea2bcbe558407ec5c)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:31.485670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:31.522231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:31.734405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a3380962fe451ea2bcbe558407ec5c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.735013 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a3380962fe451ea2bcbe558407ec5c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.818004 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a3380962fe451ea2bcbe558407ec5c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.825489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a3380962fe451ea2bcbe558407ec5c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.837310 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-abe17736-72d9-4d13-ba3c-e2dcb198293c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a3380962fe451ea2bcbe558407ec5c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:31.844108 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1386/2744] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:31 2022] DELETE /identity/v3/users/16a3380962fe451ea2bcbe558407ec5c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:32.334034 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:32.335560 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:32.349819 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d7a49-21d4-4007-a7fb-0af0316c8c8a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:32.409873 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:32.410497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:32.410894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:32.411267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] PATH_INFO: `/v3/projects/1c96990260e446a399f0e3a845ed60a6` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:32.412549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c96990260e446a399f0e3a845ed60a6)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:32.414560 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:32.516440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:32.528643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c96990260e446a399f0e3a845ed60a6', 'request_id': 'req-eb3fabe6-9164-49c7-b467-aaaddb33fe46'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:32.625001 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c96990260e446a399f0e3a845ed60a6', 'request_id': 'req-eb3fabe6-9164-49c7-b467-aaaddb33fe46'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:32.632371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c96990260e446a399f0e3a845ed60a6', 'request_id': 'req-eb3fabe6-9164-49c7-b467-aaaddb33fe46'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:32.646681 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] The token cache is being invalidate because project 1c96990260e446a399f0e3a845ed60a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:32.647124 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:32.647912 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-eb3fabe6-9164-49c7-b467-aaaddb33fe46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:32.650321 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1359/2745] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:32 2022] DELETE /identity/v3/projects/1c96990260e446a399f0e3a845ed60a6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:32.658171 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9489ecb-63fb-47e1-b755-6f79c0075724 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:32.658939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9489ecb-63fb-47e1-b755-6f79c0075724 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:32.658939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9489ecb-63fb-47e1-b755-6f79c0075724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:32.679958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d9489ecb-63fb-47e1-b755-6f79c0075724 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:32.681238 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d9489ecb-63fb-47e1-b755-6f79c0075724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:32.746664 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1387/2746] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:32.755136 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21993-e728-4a5a-8184-beefc402bf69 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:32.755643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21993-e728-4a5a-8184-beefc402bf69 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:32.756112 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21993-e728-4a5a-8184-beefc402bf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:32.776300 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-0ab21993-e728-4a5a-8184-beefc402bf69 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:32.777532 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-0ab21993-e728-4a5a-8184-beefc402bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:32.786909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1360/2747] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:01:32.805636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9489ecb-63fb-47e1-b755-6f79c0075724 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:32.816574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9489ecb-63fb-47e1-b755-6f79c0075724 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:32.821249 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae0c9b-e888-4a67-997e-ae3f4e554b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:32.821860 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae0c9b-e888-4a67-997e-ae3f4e554b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:32.822145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae0c9b-e888-4a67-997e-ae3f4e554b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:32.823045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae0c9b-e888-4a67-997e-ae3f4e554b0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:32.823712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae0c9b-e888-4a67-997e-ae3f4e554b0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:01:32.824116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae0c9b-e888-4a67-997e-ae3f4e554b0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:32.826037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae0c9b-e888-4a67-997e-ae3f4e554b0c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:32.838001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1388/2748] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:01:32.843200 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab21993-e728-4a5a-8184-beefc402bf69 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:32.852624 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab21993-e728-4a5a-8184-beefc402bf69 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:32.858860 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77a5e1-ed3c-4ce3-957c-f6afa6024f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:32.859468 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77a5e1-ed3c-4ce3-957c-f6afa6024f5f admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:32.859752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77a5e1-ed3c-4ce3-957c-f6afa6024f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:32.860028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77a5e1-ed3c-4ce3-957c-f6afa6024f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:32.860656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77a5e1-ed3c-4ce3-957c-f6afa6024f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:32.862722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77a5e1-ed3c-4ce3-957c-f6afa6024f5f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:32.979139 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1361/2749] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:32 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:01:32.983983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae0c9b-e888-4a67-997e-ae3f4e554b0c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:32.993894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae0c9b-e888-4a67-997e-ae3f4e554b0c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:33.052597 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfafcb3-0928-4e0f-9000-9abbf2151d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:33.053126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfafcb3-0928-4e0f-9000-9abbf2151d9d admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.053499 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfafcb3-0928-4e0f-9000-9abbf2151d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.053777 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfafcb3-0928-4e0f-9000-9abbf2151d9d admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.054404 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfafcb3-0928-4e0f-9000-9abbf2151d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:33.056355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfafcb3-0928-4e0f-9000-9abbf2151d9d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:33.161489 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1389/2750] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:32 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:33.165558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77a5e1-ed3c-4ce3-957c-f6afa6024f5f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:33.176211 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77a5e1-ed3c-4ce3-957c-f6afa6024f5f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:33.181041 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2153d-b3ed-4b6e-9bd1-b26d895a1d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:33.181636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2153d-b3ed-4b6e-9bd1-b26d895a1d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.182007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2153d-b3ed-4b6e-9bd1-b26d895a1d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.182286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2153d-b3ed-4b6e-9bd1-b26d895a1d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.182988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2153d-b3ed-4b6e-9bd1-b26d895a1d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:33.184562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2153d-b3ed-4b6e-9bd1-b26d895a1d7a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:33.195236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1362/2751] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:33.199218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfafcb3-0928-4e0f-9000-9abbf2151d9d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:33.209877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfafcb3-0928-4e0f-9000-9abbf2151d9d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:33.214552 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470181c-67b4-4d86-a297-f29ab83ca4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:33.215150 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470181c-67b4-4d86-a297-f29ab83ca4fa admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.215435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470181c-67b4-4d86-a297-f29ab83ca4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.215711 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470181c-67b4-4d86-a297-f29ab83ca4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.216504 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470181c-67b4-4d86-a297-f29ab83ca4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:33.218263 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470181c-67b4-4d86-a297-f29ab83ca4fa admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:33.228858 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1390/2752] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:33.234040 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2153d-b3ed-4b6e-9bd1-b26d895a1d7a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:33.245291 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2153d-b3ed-4b6e-9bd1-b26d895a1d7a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:33.251131 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37e6a6-64c8-49ab-ba17-035f16155fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:33.251781 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37e6a6-64c8-49ab-ba17-035f16155fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.252068 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37e6a6-64c8-49ab-ba17-035f16155fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.252609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37e6a6-64c8-49ab-ba17-035f16155fbf admin admin] PATH_INFO: `/v3/projects/89fc469736b54be484a0dea816f5337d/users/520cc220842e4cf083efa9f90aa2d5f0/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.268949 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37e6a6-64c8-49ab-ba17-035f16155fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=89fc469736b54be484a0dea816f5337d, user_id=520cc220842e4cf083efa9f90aa2d5f0, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:33.271758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37e6a6-64c8-49ab-ba17-035f16155fbf admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:33.337806 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1363/2753] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:33 2022] PUT /identity/v3/projects/89fc469736b54be484a0dea816f5337d/users/520cc220842e4cf083efa9f90aa2d5f0/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:33.349837 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02982-9acf-4d4c-bd4d-1c120fc630cb None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.350128 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02982-9acf-4d4c-bd4d-1c120fc630cb None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.350429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02982-9acf-4d4c-bd4d-1c120fc630cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.359668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:33.381117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-c7b02982-9acf-4d4c-bd4d-1c120fc630cb None None] MFA Rules not processed for user `520cc220842e4cf083efa9f90aa2d5f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:33.382383 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c7b02982-9acf-4d4c-bd4d-1c120fc630cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:33.448254 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1391/2754] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:33.453889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37e6a6-64c8-49ab-ba17-035f16155fbf admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:33.519279 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37e6a6-64c8-49ab-ba17-035f16155fbf admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:33.524070 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f162f-3453-4456-a673-ade32264459d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:33.524703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f162f-3453-4456-a673-ade32264459d admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.524988 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f162f-3453-4456-a673-ade32264459d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.526094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f162f-3453-4456-a673-ade32264459d admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.526094 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f162f-3453-4456-a673-ade32264459d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:33.527807 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f162f-3453-4456-a673-ade32264459d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:33.609212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1364/2755] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:33 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:33.617103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b02982-9acf-4d4c-bd4d-1c120fc630cb None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:33.624898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b02982-9acf-4d4c-bd4d-1c120fc630cb None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:33.682959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5520bdf-1b0b-4461-97f4-e97811387008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:33.683630 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5520bdf-1b0b-4461-97f4-e97811387008 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.683859 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5520bdf-1b0b-4461-97f4-e97811387008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.684138 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5520bdf-1b0b-4461-97f4-e97811387008 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.684780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5520bdf-1b0b-4461-97f4-e97811387008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:33.686803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5520bdf-1b0b-4461-97f4-e97811387008 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:33.789819 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1392/2756] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:33 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:33.794075 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f162f-3453-4456-a673-ade32264459d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:33.805263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f162f-3453-4456-a673-ade32264459d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:33.810269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f208879-b867-4614-83c7-c5cd7abbd2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:33.811021 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f208879-b867-4614-83c7-c5cd7abbd2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.811249 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f208879-b867-4614-83c7-c5cd7abbd2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.811528 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f208879-b867-4614-83c7-c5cd7abbd2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.812181 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f208879-b867-4614-83c7-c5cd7abbd2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:33.813805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f208879-b867-4614-83c7-c5cd7abbd2cd admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:33.824455 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1365/2757] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:33.828660 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5520bdf-1b0b-4461-97f4-e97811387008 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:33.838616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5520bdf-1b0b-4461-97f4-e97811387008 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:33.844625 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87684ccf-8e1a-4b65-a730-d925326ee01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:33.845224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87684ccf-8e1a-4b65-a730-d925326ee01b admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.845506 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87684ccf-8e1a-4b65-a730-d925326ee01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.845783 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87684ccf-8e1a-4b65-a730-d925326ee01b admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.846433 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87684ccf-8e1a-4b65-a730-d925326ee01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:33.848069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87684ccf-8e1a-4b65-a730-d925326ee01b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:33.859193 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1393/2758] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:33.863292 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f208879-b867-4614-83c7-c5cd7abbd2cd admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:33.873404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f208879-b867-4614-83c7-c5cd7abbd2cd admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:33.878297 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c6bd0-d36f-42ee-9d21-a6edde33d491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:33.878979 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c6bd0-d36f-42ee-9d21-a6edde33d491 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:33.879258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c6bd0-d36f-42ee-9d21-a6edde33d491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:33.879531 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c6bd0-d36f-42ee-9d21-a6edde33d491 admin admin] PATH_INFO: `/v3/projects/a0df4cc72a1b4f31927369f6b1c935de/users/b2f75a1950a7434fb7f5e920152cbc03/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:33.896932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c6bd0-d36f-42ee-9d21-a6edde33d491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0df4cc72a1b4f31927369f6b1c935de, user_id=b2f75a1950a7434fb7f5e920152cbc03, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:33.901848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c6bd0-d36f-42ee-9d21-a6edde33d491 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:33.960722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1366/2759] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:33 2022] PUT /identity/v3/projects/a0df4cc72a1b4f31927369f6b1c935de/users/b2f75a1950a7434fb7f5e920152cbc03/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:34.028664 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dd184-9f7c-46b8-8bb6-3bce2eb24384 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:34.029737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dd184-9f7c-46b8-8bb6-3bce2eb24384 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:34.030050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dd184-9f7c-46b8-8bb6-3bce2eb24384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:34.057659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-f83dd184-9f7c-46b8-8bb6-3bce2eb24384 None None] MFA Rules not processed for user `b2f75a1950a7434fb7f5e920152cbc03`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:34.058904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f83dd184-9f7c-46b8-8bb6-3bce2eb24384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:34.121735 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1394/2760] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:01:34.126772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c6bd0-d36f-42ee-9d21-a6edde33d491 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:34.136014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:34.190011 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c6bd0-d36f-42ee-9d21-a6edde33d491 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:34.194677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367be19f-cf80-4f1a-bfc3-cdddd369d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:34.195308 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367be19f-cf80-4f1a-bfc3-cdddd369d5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:34.195589 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367be19f-cf80-4f1a-bfc3-cdddd369d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:34.195864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367be19f-cf80-4f1a-bfc3-cdddd369d5d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:34.196483 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367be19f-cf80-4f1a-bfc3-cdddd369d5d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:34.198425 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367be19f-cf80-4f1a-bfc3-cdddd369d5d7 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:34.297237 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1367/2761] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:34 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:34.303710 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dd184-9f7c-46b8-8bb6-3bce2eb24384 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:34.309221 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:34.315145 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dd184-9f7c-46b8-8bb6-3bce2eb24384 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:34.370802 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb7aaf-7322-4612-87c1-b6a31628cb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:34.371405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb7aaf-7322-4612-87c1-b6a31628cb90 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:34.371685 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb7aaf-7322-4612-87c1-b6a31628cb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:34.372005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb7aaf-7322-4612-87c1-b6a31628cb90 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:34.373126 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb7aaf-7322-4612-87c1-b6a31628cb90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:34.375069 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb7aaf-7322-4612-87c1-b6a31628cb90 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:34.458456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1395/2762] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:34 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:34.462660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367be19f-cf80-4f1a-bfc3-cdddd369d5d7 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:34.473457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367be19f-cf80-4f1a-bfc3-cdddd369d5d7 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:34.478234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e54eaa-4835-46ae-9e70-b2560691420b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:34.478871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e54eaa-4835-46ae-9e70-b2560691420b admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:34.479159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e54eaa-4835-46ae-9e70-b2560691420b admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:34.479479 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e54eaa-4835-46ae-9e70-b2560691420b admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:34.480157 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e54eaa-4835-46ae-9e70-b2560691420b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:34.481759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e54eaa-4835-46ae-9e70-b2560691420b admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:34.492686 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1368/2763] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:34.496989 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb7aaf-7322-4612-87c1-b6a31628cb90 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:34.507875 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb7aaf-7322-4612-87c1-b6a31628cb90 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:34.513033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691a602-04c0-4ffd-a2fe-37a08a73e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:34.514825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691a602-04c0-4ffd-a2fe-37a08a73e75f admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:34.515189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691a602-04c0-4ffd-a2fe-37a08a73e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:34.515545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691a602-04c0-4ffd-a2fe-37a08a73e75f admin admin] PATH_INFO: `/v3/projects/fea7d597cf2b48f0b782d917ed86ad89/users/3096bbdd48774788827208c6312fd5cb/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:34.532690 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691a602-04c0-4ffd-a2fe-37a08a73e75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea7d597cf2b48f0b782d917ed86ad89, user_id=3096bbdd48774788827208c6312fd5cb, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:34.535414 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691a602-04c0-4ffd-a2fe-37a08a73e75f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:34.606662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1396/2764] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:34 2022] PUT /identity/v3/projects/fea7d597cf2b48f0b782d917ed86ad89/users/3096bbdd48774788827208c6312fd5cb/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:34.611221 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e54eaa-4835-46ae-9e70-b2560691420b admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:34.621338 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e54eaa-4835-46ae-9e70-b2560691420b admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:34.679027 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e397051-06f1-41f9-ad40-fb21ff6d1771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:34.679635 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e397051-06f1-41f9-ad40-fb21ff6d1771 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:34.679915 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e397051-06f1-41f9-ad40-fb21ff6d1771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:34.680190 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e397051-06f1-41f9-ad40-fb21ff6d1771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:34.680835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e397051-06f1-41f9-ad40-fb21ff6d1771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:34.682428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e397051-06f1-41f9-ad40-fb21ff6d1771 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:34.692780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1369/2765] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:34.697025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691a602-04c0-4ffd-a2fe-37a08a73e75f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:34.708302 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691a602-04c0-4ffd-a2fe-37a08a73e75f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:34.712898 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3c99e-a266-49b4-9ef9-26fe6fbe8092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:34.713569 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3c99e-a266-49b4-9ef9-26fe6fbe8092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:34.713849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3c99e-a266-49b4-9ef9-26fe6fbe8092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:34.714122 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3c99e-a266-49b4-9ef9-26fe6fbe8092 admin admin] PATH_INFO: `/v3/projects/fea7d597cf2b48f0b782d917ed86ad89/users/3096bbdd48774788827208c6312fd5cb/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:34.716113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3c99e-a266-49b4-9ef9-26fe6fbe8092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea7d597cf2b48f0b782d917ed86ad89, user_id=3096bbdd48774788827208c6312fd5cb, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:34.718786 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3c99e-a266-49b4-9ef9-26fe6fbe8092 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:34.728796 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1397/2766] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:34 2022] PUT /identity/v3/projects/fea7d597cf2b48f0b782d917ed86ad89/users/3096bbdd48774788827208c6312fd5cb/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:34.739587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3845b9-7e5d-4f29-9503-01d323b200d4 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:34.739878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3845b9-7e5d-4f29-9503-01d323b200d4 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:34.740177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3845b9-7e5d-4f29-9503-01d323b200d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:34.768232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-9f3845b9-7e5d-4f29-9503-01d323b200d4 None None] MFA Rules not processed for user `3096bbdd48774788827208c6312fd5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:34.769444 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9f3845b9-7e5d-4f29-9503-01d323b200d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:34.838638 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1370/2767] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:34.849937 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3c99e-a266-49b4-9ef9-26fe6fbe8092 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:34.851712 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-04c3c99e-a266-49b4-9ef9-26fe6fbe8092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:34.910440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3c99e-a266-49b4-9ef9-26fe6fbe8092 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:34.914287 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10053c-a144-44c0-84a4-09d90bccdad0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:34.914938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10053c-a144-44c0-84a4-09d90bccdad0 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:34.915220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10053c-a144-44c0-84a4-09d90bccdad0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:34.915493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10053c-a144-44c0-84a4-09d90bccdad0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:34.925034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10053c-a144-44c0-84a4-09d90bccdad0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:34.926475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10053c-a144-44c0-84a4-09d90bccdad0 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:34.936012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1398/2768] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:36.094887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3845b9-7e5d-4f29-9503-01d323b200d4 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:36.099955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:36.105585 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3845b9-7e5d-4f29-9503-01d323b200d4 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:36.110050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535330af-2bf4-46ab-a75a-cdbda83b0fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:36.110694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535330af-2bf4-46ab-a75a-cdbda83b0fba service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:36.110977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535330af-2bf4-46ab-a75a-cdbda83b0fba service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:36.111339 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535330af-2bf4-46ab-a75a-cdbda83b0fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:36.169880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535330af-2bf4-46ab-a75a-cdbda83b0fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:36.171729 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535330af-2bf4-46ab-a75a-cdbda83b0fba service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:36.180857 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1371/2769] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:44.931674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:45.478115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10053c-a144-44c0-84a4-09d90bccdad0 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:45.487834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10053c-a144-44c0-84a4-09d90bccdad0 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:45.491560 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a74c-26cc-4253-98de-1398b0316e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:45.492189 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a74c-26cc-4253-98de-1398b0316e2d service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:45.492470 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a74c-26cc-4253-98de-1398b0316e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:45.492743 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a74c-26cc-4253-98de-1398b0316e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:45.556491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275a74c-26cc-4253-98de-1398b0316e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:45.557981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275a74c-26cc-4253-98de-1398b0316e2d service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:45.573395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1399/2770] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:46.234892 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=27 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:48.099488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119b15e-5dad-48c4-8d13-c40e4c8f69d3 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:48.100101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119b15e-5dad-48c4-8d13-c40e4c8f69d3 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:48.100101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119b15e-5dad-48c4-8d13-c40e4c8f69d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:48.131392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-e119b15e-5dad-48c4-8d13-c40e4c8f69d3 None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:48.132549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-e119b15e-5dad-48c4-8d13-c40e4c8f69d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:48.197794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1372/2771] 10.222.0.82 () {58 vars in 1009 bytes} [Wed Aug 24 04:01:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:48.314344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a74c-26cc-4253-98de-1398b0316e2d service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:48.315981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1275a74c-26cc-4253-98de-1398b0316e2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:48.387339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a74c-26cc-4253-98de-1398b0316e2d service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:48.392884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e1286-43b4-4802-9997-408a219677c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:48.396522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e1286-43b4-4802-9997-408a219677c5 service swift] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:48.396522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e1286-43b4-4802-9997-408a219677c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:48.396522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e1286-43b4-4802-9997-408a219677c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:48.402030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e1286-43b4-4802-9997-408a219677c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:48.404412 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e1286-43b4-4802-9997-408a219677c5 service swift] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:48.412576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1400/2772] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 04:01:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 04:01:50.392708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119b15e-5dad-48c4-8d13-c40e4c8f69d3 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:50.478921 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119b15e-5dad-48c4-8d13-c40e4c8f69d3 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:50.484962 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:50.485706 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:50.486202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:50.486592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] PATH_INFO: `/v3/users/520cc220842e4cf083efa9f90aa2d5f0` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:50.487971 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] RBAC: Authorizing `identity:delete_user(user_id=520cc220842e4cf083efa9f90aa2d5f0)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:50.489901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:50.760770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '520cc220842e4cf083efa9f90aa2d5f0'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:50.834762 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '520cc220842e4cf083efa9f90aa2d5f0'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:50.852641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '520cc220842e4cf083efa9f90aa2d5f0'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:50.864584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '520cc220842e4cf083efa9f90aa2d5f0'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:50.866633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '520cc220842e4cf083efa9f90aa2d5f0'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:50.873835 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1373/2773] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:50 2022] DELETE /identity/v3/users/520cc220842e4cf083efa9f90aa2d5f0 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:50.879900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963c06c-3335-4690-a858-6faecae94340 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:50.880515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963c06c-3335-4690-a858-6faecae94340 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:50.880515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963c06c-3335-4690-a858-6faecae94340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:50.904616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-7963c06c-3335-4690-a858-6faecae94340 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:50.906105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7963c06c-3335-4690-a858-6faecae94340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:50.978840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1401/2774] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:01:50.990073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:50.991800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:51.004947 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:51.070005 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6516440-8ded-4dd7-832f-04e59335352f admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:51.073719 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c926d-83eb-4bf2-894f-86c9505633b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:51.074377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c926d-83eb-4bf2-894f-86c9505633b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:51.074724 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c926d-83eb-4bf2-894f-86c9505633b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:51.075038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c926d-83eb-4bf2-894f-86c9505633b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:51.087030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c926d-83eb-4bf2-894f-86c9505633b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:51.088600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c926d-83eb-4bf2-894f-86c9505633b7 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:51.101559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1374/2775] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:51.597586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963c06c-3335-4690-a858-6faecae94340 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:51.599510 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7963c06c-3335-4690-a858-6faecae94340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:51.616005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963c06c-3335-4690-a858-6faecae94340 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:51.621154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:51.622230 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:51.622627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:51.623014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] PATH_INFO: `/v3/users/442f28e73ed9496a8c762093f31a9594` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:51.624486 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=442f28e73ed9496a8c762093f31a9594)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:51.628963 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:51.659327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:51.701048 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c926d-83eb-4bf2-894f-86c9505633b7 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:51.703440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-313c926d-83eb-4bf2-894f-86c9505633b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:51.726713 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c926d-83eb-4bf2-894f-86c9505633b7 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:51.733222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:51.736849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:51.736849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:51.736849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] PATH_INFO: `/v3/projects/89fc469736b54be484a0dea816f5337d` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:51.736849 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89fc469736b54be484a0dea816f5337d)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:51.753170 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:51.901616 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '442f28e73ed9496a8c762093f31a9594'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:51.904557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '442f28e73ed9496a8c762093f31a9594'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:51.921230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89fc469736b54be484a0dea816f5337d', 'request_id': 'req-9ae746a2-a5bd-4903-aca3-ebffae69f917'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:51.989463 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '442f28e73ed9496a8c762093f31a9594'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:51.999509 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '442f28e73ed9496a8c762093f31a9594'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.033943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-7045ca70-fa66-493b-831c-a3966906e8f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '442f28e73ed9496a8c762093f31a9594'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.042564 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1402/2776] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:51 2022] DELETE /identity/v3/users/442f28e73ed9496a8c762093f31a9594 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:52.048350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189fbe9-30f8-4d7d-ad60-d455c2e1234e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:52.048889 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189fbe9-30f8-4d7d-ad60-d455c2e1234e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:52.049385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189fbe9-30f8-4d7d-ad60-d455c2e1234e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:52.074927 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-2189fbe9-30f8-4d7d-ad60-d455c2e1234e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:52.076244 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2189fbe9-30f8-4d7d-ad60-d455c2e1234e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:52.083183 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89fc469736b54be484a0dea816f5337d', 'request_id': 'req-9ae746a2-a5bd-4903-aca3-ebffae69f917'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.094130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89fc469736b54be484a0dea816f5337d', 'request_id': 'req-9ae746a2-a5bd-4903-aca3-ebffae69f917'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.118161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] The token cache is being invalidate because project 89fc469736b54be484a0dea816f5337d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:52.119145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.119987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.125108 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1375/2777] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:51 2022] DELETE /identity/v3/projects/89fc469736b54be484a0dea816f5337d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:52.133771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:52.135830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:52.206475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1403/2778] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:01:52.219629 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189fbe9-30f8-4d7d-ad60-d455c2e1234e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:52.228835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-2189fbe9-30f8-4d7d-ad60-d455c2e1234e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:52.236778 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae746a2-a5bd-4903-aca3-ebffae69f917 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:52.242543 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:52.243598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:52.243986 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:52.244392 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] PATH_INFO: `/v3/users/b2f75a1950a7434fb7f5e920152cbc03` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:52.245867 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f75a1950a7434fb7f5e920152cbc03)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:52.247848 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:52.310087 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189fbe9-30f8-4d7d-ad60-d455c2e1234e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:52.314407 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da43d45-de36-4642-a0f6-be960553dda7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:52.315127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da43d45-de36-4642-a0f6-be960553dda7 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:52.315510 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da43d45-de36-4642-a0f6-be960553dda7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:52.316001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da43d45-de36-4642-a0f6-be960553dda7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:52.331071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da43d45-de36-4642-a0f6-be960553dda7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:52.332758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da43d45-de36-4642-a0f6-be960553dda7 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:52.346091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1404/2779] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:52.389983 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:52.410527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f75a1950a7434fb7f5e920152cbc03'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.502049 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f75a1950a7434fb7f5e920152cbc03'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.517861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f75a1950a7434fb7f5e920152cbc03'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.525576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f75a1950a7434fb7f5e920152cbc03'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.526331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f75a1950a7434fb7f5e920152cbc03'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:52.533019 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1376/2780] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:52 2022] DELETE /identity/v3/users/b2f75a1950a7434fb7f5e920152cbc03 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:52.907565 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da43d45-de36-4642-a0f6-be960553dda7 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:52.909116 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1da43d45-de36-4642-a0f6-be960553dda7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:52.938354 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da43d45-de36-4642-a0f6-be960553dda7 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:52.993177 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:53.027493 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:53.028622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:53.028919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:53.028919 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] PATH_INFO: `/v3/projects/9d25382b4a8e404bbc5a823cdf6f624e` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:53.036288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d25382b4a8e404bbc5a823cdf6f624e)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:53.038239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:53.165758 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:53.167403 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:53.181302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025f33a-3917-4c49-9519-80155a47e353 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:53.189745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:53.192889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:53.192889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:53.192889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] PATH_INFO: `/v3/projects/a0df4cc72a1b4f31927369f6b1c935de` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:53.192889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0df4cc72a1b4f31927369f6b1c935de)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:53.194178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:53.225431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d25382b4a8e404bbc5a823cdf6f624e', 'request_id': 'req-95991851-4261-47f3-bbab-5d794d2817ee'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.234491 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d25382b4a8e404bbc5a823cdf6f624e', 'request_id': 'req-95991851-4261-47f3-bbab-5d794d2817ee'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.327509 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d25382b4a8e404bbc5a823cdf6f624e', 'request_id': 'req-95991851-4261-47f3-bbab-5d794d2817ee'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.338761 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0df4cc72a1b4f31927369f6b1c935de', 'request_id': 'req-391b34ee-573f-445f-abed-c867ce38ac13'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.355719 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] The token cache is being invalidate because project 9d25382b4a8e404bbc5a823cdf6f624e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:53.360822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.360822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-95991851-4261-47f3-bbab-5d794d2817ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.360822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1405/2781] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:52 2022] DELETE /identity/v3/projects/9d25382b4a8e404bbc5a823cdf6f624e => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:53.365472 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458eb9a-e0f9-41cf-8540-0ffdea7304dd None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:53.365763 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458eb9a-e0f9-41cf-8540-0ffdea7304dd None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:53.366043 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458eb9a-e0f9-41cf-8540-0ffdea7304dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:53.392740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-8458eb9a-e0f9-41cf-8540-0ffdea7304dd None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:53.410231 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8458eb9a-e0f9-41cf-8540-0ffdea7304dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:53.428902 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0df4cc72a1b4f31927369f6b1c935de', 'request_id': 'req-391b34ee-573f-445f-abed-c867ce38ac13'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.438345 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0df4cc72a1b4f31927369f6b1c935de', 'request_id': 'req-391b34ee-573f-445f-abed-c867ce38ac13'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.453450 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] The token cache is being invalidate because project a0df4cc72a1b4f31927369f6b1c935de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:53.453748 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.454485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.456369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1377/2782] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:53 2022] DELETE /identity/v3/projects/a0df4cc72a1b4f31927369f6b1c935de => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:53.460412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:53.464044 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:53.494077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1406/2783] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:53.579206 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b34ee-573f-445f-abed-c867ce38ac13 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:53.582071 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2a78a-28bf-4953-a2bd-60b2139dd619 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:53.582914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2a78a-28bf-4953-a2bd-60b2139dd619 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:53.583624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2a78a-28bf-4953-a2bd-60b2139dd619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:53.584895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:53.585722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:53.586229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:53.586745 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] PATH_INFO: `/v3/users/3096bbdd48774788827208c6312fd5cb` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:53.589598 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3096bbdd48774788827208c6312fd5cb)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:53.594811 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:53.635936 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-a7f2a78a-28bf-4953-a2bd-60b2139dd619 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:53.637877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a7f2a78a-28bf-4953-a2bd-60b2139dd619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:53.650555 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1407/2784] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:53.658057 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2a78a-28bf-4953-a2bd-60b2139dd619 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:53.670225 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2a78a-28bf-4953-a2bd-60b2139dd619 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:53.676324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c13862-f0bc-48bf-9ccd-d991bdee9b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:53.677163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c13862-f0bc-48bf-9ccd-d991bdee9b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:53.677465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c13862-f0bc-48bf-9ccd-d991bdee9b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:53.677798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c13862-f0bc-48bf-9ccd-d991bdee9b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:53.678781 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c13862-f0bc-48bf-9ccd-d991bdee9b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:01:53.679204 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c13862-f0bc-48bf-9ccd-d991bdee9b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:53.681277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c13862-f0bc-48bf-9ccd-d991bdee9b1d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:53.698585 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1408/2785] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:01:53.701435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c13862-f0bc-48bf-9ccd-d991bdee9b1d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:53.717035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c13862-f0bc-48bf-9ccd-d991bdee9b1d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:53.725048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb76a2-fdcc-4d1c-898f-0e1a6b18532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:53.725670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb76a2-fdcc-4d1c-898f-0e1a6b18532f admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:53.725964 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb76a2-fdcc-4d1c-898f-0e1a6b18532f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:53.726226 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb76a2-fdcc-4d1c-898f-0e1a6b18532f admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:53.727052 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb76a2-fdcc-4d1c-898f-0e1a6b18532f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:53.729084 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb76a2-fdcc-4d1c-898f-0e1a6b18532f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:53.782134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:53.790077 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3096bbdd48774788827208c6312fd5cb'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.907168 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1409/2786] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:53 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:53.908586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb76a2-fdcc-4d1c-898f-0e1a6b18532f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:53.922554 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3096bbdd48774788827208c6312fd5cb'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.945495 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3096bbdd48774788827208c6312fd5cb'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.959028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3096bbdd48774788827208c6312fd5cb'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.960633 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3096bbdd48774788827208c6312fd5cb'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:53.974588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1378/2787] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:53 2022] DELETE /identity/v3/users/3096bbdd48774788827208c6312fd5cb => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:53.982232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:54.030415 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb76a2-fdcc-4d1c-898f-0e1a6b18532f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:54.121014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0275ad4-9ed2-4040-9536-daa97bc821f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:54.122293 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0275ad4-9ed2-4040-9536-daa97bc821f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:54.122671 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0275ad4-9ed2-4040-9536-daa97bc821f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:54.122960 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0275ad4-9ed2-4040-9536-daa97bc821f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:54.123729 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0275ad4-9ed2-4040-9536-daa97bc821f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:54.125840 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0275ad4-9ed2-4040-9536-daa97bc821f2 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:54.470299 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1410/2788] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:53 2022] POST /identity/v3/users => generated 290 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:54.474387 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:54.475854 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:54.490839 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678eb55-d27d-4973-99e4-7414fb56ba65 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:54.495472 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f937b-c387-409a-84b1-6e1bd33c20e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:54.496080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f937b-c387-409a-84b1-6e1bd33c20e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:54.496374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f937b-c387-409a-84b1-6e1bd33c20e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:54.497235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f937b-c387-409a-84b1-6e1bd33c20e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:54.497890 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f937b-c387-409a-84b1-6e1bd33c20e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:54.499529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f937b-c387-409a-84b1-6e1bd33c20e9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:54.517511 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0275ad4-9ed2-4040-9536-daa97bc821f2 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:54.521408 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1379/2789] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:54.525273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f937b-c387-409a-84b1-6e1bd33c20e9 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:54.530705 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b0275ad4-9ed2-4040-9536-daa97bc821f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:54.544273 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f937b-c387-409a-84b1-6e1bd33c20e9 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:54.548335 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0275ad4-9ed2-4040-9536-daa97bc821f2 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:54.554317 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:54.555256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:54.555826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:54.556205 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] PATH_INFO: `/v3/projects/fea7d597cf2b48f0b782d917ed86ad89` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:54.557454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea7d597cf2b48f0b782d917ed86ad89)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:54.557671 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5f0bc-5b5a-4c69-9c21-6f5704873849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:54.558302 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f0bc-5b5a-4c69-9c21-6f5704873849 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:54.558617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f0bc-5b5a-4c69-9c21-6f5704873849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:54.558959 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f0bc-5b5a-4c69-9c21-6f5704873849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:54.560680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:54.560831 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5f0bc-5b5a-4c69-9c21-6f5704873849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:54.562617 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5f0bc-5b5a-4c69-9c21-6f5704873849 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:54.578458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1380/2790] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:54.582329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5f0bc-5b5a-4c69-9c21-6f5704873849 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:54.597286 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5f0bc-5b5a-4c69-9c21-6f5704873849 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:54.604120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b03813-9c71-41e1-a592-14f80a99e8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:54.609669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03813-9c71-41e1-a592-14f80a99e8f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:54.609669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03813-9c71-41e1-a592-14f80a99e8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:54.609669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03813-9c71-41e1-a592-14f80a99e8f5 admin admin] PATH_INFO: `/v3/projects/8d390e7ad1454a8182258cd72bea4fa5/users/34d452ce6a1245c192acf8c87651fe92/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:54.634367 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b03813-9c71-41e1-a592-14f80a99e8f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d390e7ad1454a8182258cd72bea4fa5, user_id=34d452ce6a1245c192acf8c87651fe92, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:54.639330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b03813-9c71-41e1-a592-14f80a99e8f5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:54.729013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1381/2791] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:54 2022] PUT /identity/v3/projects/8d390e7ad1454a8182258cd72bea4fa5/users/34d452ce6a1245c192acf8c87651fe92/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:01:54.735250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d190b3d-231d-42d8-af34-2333a290628a None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:54.735846 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d190b3d-231d-42d8-af34-2333a290628a None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:54.736426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d190b3d-231d-42d8-af34-2333a290628a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:54.742155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea7d597cf2b48f0b782d917ed86ad89', 'request_id': 'req-151870f3-f84a-4324-a71d-1217e1226aef'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:54.751583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea7d597cf2b48f0b782d917ed86ad89', 'request_id': 'req-151870f3-f84a-4324-a71d-1217e1226aef'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:54.765217 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-4d190b3d-231d-42d8-af34-2333a290628a None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:54.766523 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4d190b3d-231d-42d8-af34-2333a290628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:54.852106 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea7d597cf2b48f0b782d917ed86ad89', 'request_id': 'req-151870f3-f84a-4324-a71d-1217e1226aef'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:54.869844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1382/2792] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:54.873808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] The token cache is being invalidate because project fea7d597cf2b48f0b782d917ed86ad89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:01:54.874271 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:54.875171 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-151870f3-f84a-4324-a71d-1217e1226aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:54.878545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1411/2793] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:54 2022] DELETE /identity/v3/projects/fea7d597cf2b48f0b782d917ed86ad89 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:54.882797 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d190b3d-231d-42d8-af34-2333a290628a None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:54.884642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81f3b3-e4c8-408f-8ea9-f446bb2328e4 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:54.884935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81f3b3-e4c8-408f-8ea9-f446bb2328e4 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:54.885220 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81f3b3-e4c8-408f-8ea9-f446bb2328e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:54.888853 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4d190b3d-231d-42d8-af34-2333a290628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:54.919846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-eb81f3b3-e4c8-408f-8ea9-f446bb2328e4 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:54.921200 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-eb81f3b3-e4c8-408f-8ea9-f446bb2328e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:54.939295 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1412/2794] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:01:54.950495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6117941-713f-4975-9bc9-c859205da58b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:54.950856 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6117941-713f-4975-9bc9-c859205da58b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:54.951229 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6117941-713f-4975-9bc9-c859205da58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:54.974363 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-d6117941-713f-4975-9bc9-c859205da58b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:54.975788 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-d6117941-713f-4975-9bc9-c859205da58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:54.990316 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1413/2795] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:54.990553 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d190b3d-231d-42d8-af34-2333a290628a None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:54.991649 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81758f-4e20-4404-a548-4c6f0db30e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:54.991798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6117941-713f-4975-9bc9-c859205da58b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:54.992243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81758f-4e20-4404-a548-4c6f0db30e99 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:54.992536 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81758f-4e20-4404-a548-4c6f0db30e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:54.992818 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81758f-4e20-4404-a548-4c6f0db30e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:54.995382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-dc81758f-4e20-4404-a548-4c6f0db30e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:55.009925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6117941-713f-4975-9bc9-c859205da58b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:55.018968 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bd6f1-ef7d-42c2-853d-271bebd7d2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:55.019661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bd6f1-ef7d-42c2-853d-271bebd7d2bd admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:55.020080 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bd6f1-ef7d-42c2-853d-271bebd7d2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:55.020508 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bd6f1-ef7d-42c2-853d-271bebd7d2bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:55.021632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8bd6f1-ef7d-42c2-853d-271bebd7d2bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:01:55.021632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8bd6f1-ef7d-42c2-853d-271bebd7d2bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:55.023461 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8bd6f1-ef7d-42c2-853d-271bebd7d2bd admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:55.024260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81758f-4e20-4404-a548-4c6f0db30e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:55.028118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81758f-4e20-4404-a548-4c6f0db30e99 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:55.038548 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1414/2796] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:01:55.042483 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bd6f1-ef7d-42c2-853d-271bebd7d2bd admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:55.049038 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1383/2797] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:55.066610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bd6f1-ef7d-42c2-853d-271bebd7d2bd admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:55.074042 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f680be-ceec-432e-9a8e-bfcc63cee17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:55.074851 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f680be-ceec-432e-9a8e-bfcc63cee17a admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:55.075627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f680be-ceec-432e-9a8e-bfcc63cee17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:55.075627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f680be-ceec-432e-9a8e-bfcc63cee17a admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:55.076258 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f680be-ceec-432e-9a8e-bfcc63cee17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:55.078216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f680be-ceec-432e-9a8e-bfcc63cee17a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:55.217954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1415/2798] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:55 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:01:55.222463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81758f-4e20-4404-a548-4c6f0db30e99 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:55.240284 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81758f-4e20-4404-a548-4c6f0db30e99 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:55.253163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:55.321178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2501ed1-133c-44a0-9491-628ae251a3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:55.321796 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501ed1-133c-44a0-9491-628ae251a3ff admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:55.322081 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501ed1-133c-44a0-9491-628ae251a3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:55.322360 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2501ed1-133c-44a0-9491-628ae251a3ff admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:55.323030 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2501ed1-133c-44a0-9491-628ae251a3ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:55.325861 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2501ed1-133c-44a0-9491-628ae251a3ff admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:55.440731 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1384/2799] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:55 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:55.444513 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f680be-ceec-432e-9a8e-bfcc63cee17a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:55.462644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:55.470424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f680be-ceec-432e-9a8e-bfcc63cee17a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:55.480502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f7ace-e2e9-495d-b453-dbb848af06f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:55.485045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f7ace-e2e9-495d-b453-dbb848af06f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:55.485045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f7ace-e2e9-495d-b453-dbb848af06f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:55.485045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f7ace-e2e9-495d-b453-dbb848af06f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:55.485045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f7ace-e2e9-495d-b453-dbb848af06f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:55.486658 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f7ace-e2e9-495d-b453-dbb848af06f0 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:55.506603 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1416/2800] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:55.508177 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2501ed1-133c-44a0-9491-628ae251a3ff admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:55.520149 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2501ed1-133c-44a0-9491-628ae251a3ff admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:55.526263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a310f-e914-48fe-809d-ca5753144e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:55.527047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a310f-e914-48fe-809d-ca5753144e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:55.527342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a310f-e914-48fe-809d-ca5753144e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:55.527616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a310f-e914-48fe-809d-ca5753144e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:55.528944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a310f-e914-48fe-809d-ca5753144e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:55.531304 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a310f-e914-48fe-809d-ca5753144e88 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:55.546557 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1385/2801] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:01:55.548194 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f7ace-e2e9-495d-b453-dbb848af06f0 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:55.558852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f7ace-e2e9-495d-b453-dbb848af06f0 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:55.565040 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ec88e-ad0d-4d6f-b475-d6ed905b6e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:55.565706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ec88e-ad0d-4d6f-b475-d6ed905b6e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:55.566035 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ec88e-ad0d-4d6f-b475-d6ed905b6e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:55.566317 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ec88e-ad0d-4d6f-b475-d6ed905b6e23 admin admin] PATH_INFO: `/v3/projects/793d1f55c3f6474a8f9adb482b6a8231/users/7ee832f883a74fc5bb2409f8026c96fc/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:55.584985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ec88e-ad0d-4d6f-b475-d6ed905b6e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=793d1f55c3f6474a8f9adb482b6a8231, user_id=7ee832f883a74fc5bb2409f8026c96fc, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:55.589149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ec88e-ad0d-4d6f-b475-d6ed905b6e23 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:55.722647 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1417/2802] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:55 2022] PUT /identity/v3/projects/793d1f55c3f6474a8f9adb482b6a8231/users/7ee832f883a74fc5bb2409f8026c96fc/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:55.738201 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae810d-db7a-456f-a497-4c03b32ce124 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:55.738524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae810d-db7a-456f-a497-4c03b32ce124 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:55.738891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae810d-db7a-456f-a497-4c03b32ce124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:55.793013 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-2cae810d-db7a-456f-a497-4c03b32ce124 None None] MFA Rules not processed for user `7ee832f883a74fc5bb2409f8026c96fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:55.796113 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2cae810d-db7a-456f-a497-4c03b32ce124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:55.899559 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1386/2803] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:55.900956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ec88e-ad0d-4d6f-b475-d6ed905b6e23 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:56.011038 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ec88e-ad0d-4d6f-b475-d6ed905b6e23 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:56.018533 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ff495-476e-496b-8b5e-477cde3a5ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:56.019708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ff495-476e-496b-8b5e-477cde3a5ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:56.020503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ff495-476e-496b-8b5e-477cde3a5ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:56.020503 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ff495-476e-496b-8b5e-477cde3a5ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:56.020963 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ff495-476e-496b-8b5e-477cde3a5ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:56.023232 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ff495-476e-496b-8b5e-477cde3a5ed9 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:56.180626 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1418/2804] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:01:55 2022] POST /identity/v3/projects => generated 383 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:56.186118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae810d-db7a-456f-a497-4c03b32ce124 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:56.198298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae810d-db7a-456f-a497-4c03b32ce124 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:56.291334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ea737-ed75-4cee-8b18-87e0c135e98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:56.292115 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ea737-ed75-4cee-8b18-87e0c135e98d admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:56.292547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ea737-ed75-4cee-8b18-87e0c135e98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:56.292927 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ea737-ed75-4cee-8b18-87e0c135e98d admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:56.294418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ea737-ed75-4cee-8b18-87e0c135e98d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:56.296458 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ea737-ed75-4cee-8b18-87e0c135e98d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:56.420083 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1387/2805] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:01:56 2022] POST /identity/v3/users => generated 298 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:01:56.421273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ff495-476e-496b-8b5e-477cde3a5ed9 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:56.435405 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ff495-476e-496b-8b5e-477cde3a5ed9 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:56.441153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd133da-a7ac-4f0a-a83f-4af3d67e6a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:56.441780 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd133da-a7ac-4f0a-a83f-4af3d67e6a96 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:56.444901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd133da-a7ac-4f0a-a83f-4af3d67e6a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:56.444901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd133da-a7ac-4f0a-a83f-4af3d67e6a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:56.444901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd133da-a7ac-4f0a-a83f-4af3d67e6a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:56.444901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd133da-a7ac-4f0a-a83f-4af3d67e6a96 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:56.458014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1419/2806] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:56.462813 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ea737-ed75-4cee-8b18-87e0c135e98d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:56.477324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ea737-ed75-4cee-8b18-87e0c135e98d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:56.486060 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e58eac-af45-492b-808b-8852efcceb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:56.487080 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e58eac-af45-492b-808b-8852efcceb32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:56.487780 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e58eac-af45-492b-808b-8852efcceb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:56.488191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e58eac-af45-492b-808b-8852efcceb32 admin admin] PATH_INFO: `/v3/projects/a2a42db497fe4c3a999749d65ee98906/users/6097b1efdce848b1b597b19be92ba3f2/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:56.500641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:56.513683 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e58eac-af45-492b-808b-8852efcceb32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a42db497fe4c3a999749d65ee98906, user_id=6097b1efdce848b1b597b19be92ba3f2, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:56.516825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e58eac-af45-492b-808b-8852efcceb32 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:56.614822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1388/2807] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:56 2022] PUT /identity/v3/projects/a2a42db497fe4c3a999749d65ee98906/users/6097b1efdce848b1b597b19be92ba3f2/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:01:56.619088 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd133da-a7ac-4f0a-a83f-4af3d67e6a96 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:56.632428 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd133da-a7ac-4f0a-a83f-4af3d67e6a96 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:56.711098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e5db0-30c5-494e-aed6-51fad0885671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:56.711912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e5db0-30c5-494e-aed6-51fad0885671 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:56.712319 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e5db0-30c5-494e-aed6-51fad0885671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:56.712843 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e5db0-30c5-494e-aed6-51fad0885671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:56.713965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e5db0-30c5-494e-aed6-51fad0885671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:56.716149 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e5db0-30c5-494e-aed6-51fad0885671 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:56.734545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1420/2808] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:01:56.735895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e58eac-af45-492b-808b-8852efcceb32 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:56.756219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e58eac-af45-492b-808b-8852efcceb32 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:56.764074 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f4cc-d0d4-4482-9153-b1a366fe8d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:56.764763 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f4cc-d0d4-4482-9153-b1a366fe8d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:56.765046 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f4cc-d0d4-4482-9153-b1a366fe8d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:56.765328 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f4cc-d0d4-4482-9153-b1a366fe8d0c admin admin] PATH_INFO: `/v3/projects/a2a42db497fe4c3a999749d65ee98906/users/6097b1efdce848b1b597b19be92ba3f2/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:56.767830 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379f4cc-d0d4-4482-9153-b1a366fe8d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a42db497fe4c3a999749d65ee98906, user_id=6097b1efdce848b1b597b19be92ba3f2, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:56.770904 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379f4cc-d0d4-4482-9153-b1a366fe8d0c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:56.783991 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1389/2809] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:01:56 2022] PUT /identity/v3/projects/a2a42db497fe4c3a999749d65ee98906/users/6097b1efdce848b1b597b19be92ba3f2/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:56.795628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb89fd5-c296-43f4-8cea-2e47c2ce0af0 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:56.796105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb89fd5-c296-43f4-8cea-2e47c2ce0af0 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:56.796396 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb89fd5-c296-43f4-8cea-2e47c2ce0af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:56.833985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-cfb89fd5-c296-43f4-8cea-2e47c2ce0af0 None None] MFA Rules not processed for user `6097b1efdce848b1b597b19be92ba3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:56.835397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-cfb89fd5-c296-43f4-8cea-2e47c2ce0af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:56.862677 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:56.974142 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1421/2810] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:56.987982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f4cc-d0d4-4482-9153-b1a366fe8d0c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:56.990127 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-2379f4cc-d0d4-4482-9153-b1a366fe8d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:57.101953 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f4cc-d0d4-4482-9153-b1a366fe8d0c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:57.109764 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2890ff8-ffea-4c26-b240-62e0be639f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:57.112055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2890ff8-ffea-4c26-b240-62e0be639f9f service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:57.112679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2890ff8-ffea-4c26-b240-62e0be639f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:57.112679 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2890ff8-ffea-4c26-b240-62e0be639f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:57.136799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2890ff8-ffea-4c26-b240-62e0be639f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:57.139310 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2890ff8-ffea-4c26-b240-62e0be639f9f service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:57.155469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1390/2811] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:01:57.316384 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb89fd5-c296-43f4-8cea-2e47c2ce0af0 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:57.327929 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb89fd5-c296-43f4-8cea-2e47c2ce0af0 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:57.332209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1f753-6c94-4a8f-a118-bd5b6f4953ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:57.332884 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1f753-6c94-4a8f-a118-bd5b6f4953ac service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:57.333218 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1f753-6c94-4a8f-a118-bd5b6f4953ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:57.334311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1f753-6c94-4a8f-a118-bd5b6f4953ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:57.411891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1f753-6c94-4a8f-a118-bd5b6f4953ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:57.413659 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1f753-6c94-4a8f-a118-bd5b6f4953ac service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:57.439742 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1422/2812] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:58.323567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2890ff8-ffea-4c26-b240-62e0be639f9f service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:58.412880 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2890ff8-ffea-4c26-b240-62e0be639f9f service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:58.418257 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:58.419841 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:58.420445 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:58.421210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] PATH_INFO: `/v3/users/7ee832f883a74fc5bb2409f8026c96fc` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:58.422578 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee832f883a74fc5bb2409f8026c96fc)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:58.425143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:58.639599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee832f883a74fc5bb2409f8026c96fc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:58.703874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee832f883a74fc5bb2409f8026c96fc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:58.727799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee832f883a74fc5bb2409f8026c96fc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:58.737073 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:01:58.741018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee832f883a74fc5bb2409f8026c96fc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:58.742120 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee832f883a74fc5bb2409f8026c96fc'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:01:58.759236 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1391/2813] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:01:58 2022] DELETE /identity/v3/users/7ee832f883a74fc5bb2409f8026c96fc => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:01:58.760826 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a471bb-f86f-4e32-a3ed-0d7f9033f151 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:58.761322 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a471bb-f86f-4e32-a3ed-0d7f9033f151 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:58.761423 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a471bb-f86f-4e32-a3ed-0d7f9033f151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:58.794935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-07a471bb-f86f-4e32-a3ed-0d7f9033f151 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:01:58.798914 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-07a471bb-f86f-4e32-a3ed-0d7f9033f151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:58.901978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1423/2814] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:01:58.914165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:58.916086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:59.023680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8e055-ccbc-48d7-8a25-26eb90203280 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:59.029737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea180175-0d5b-4c08-ba7e-59b99873f6bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:59.030433 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea180175-0d5b-4c08-ba7e-59b99873f6bf service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:59.030752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea180175-0d5b-4c08-ba7e-59b99873f6bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:59.031036 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea180175-0d5b-4c08-ba7e-59b99873f6bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:59.047231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea180175-0d5b-4c08-ba7e-59b99873f6bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:59.049111 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea180175-0d5b-4c08-ba7e-59b99873f6bf service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:01:59.073329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1392/2815] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:01:59.873427 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a471bb-f86f-4e32-a3ed-0d7f9033f151 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:01:59.874980 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-07a471bb-f86f-4e32-a3ed-0d7f9033f151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:01:59.909199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a471bb-f86f-4e32-a3ed-0d7f9033f151 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:01:59.916468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:01:59.920223 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:01:59.920787 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:01:59.921495 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] PATH_INFO: `/v3/projects/793d1f55c3f6474a8f9adb482b6a8231` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:01:59.923822 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=793d1f55c3f6474a8f9adb482b6a8231)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:01:59.928850 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:00.106853 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '793d1f55c3f6474a8f9adb482b6a8231', 'request_id': 'req-c8c64319-4ea3-414c-9390-fa1c4122e2bd'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.109502 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:00.126696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '793d1f55c3f6474a8f9adb482b6a8231', 'request_id': 'req-c8c64319-4ea3-414c-9390-fa1c4122e2bd'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.188103 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '793d1f55c3f6474a8f9adb482b6a8231', 'request_id': 'req-c8c64319-4ea3-414c-9390-fa1c4122e2bd'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.208737 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] The token cache is being invalidate because project 793d1f55c3f6474a8f9adb482b6a8231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:02:00.209327 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.210465 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.213345 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1424/2816] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:01:59 2022] DELETE /identity/v3/projects/793d1f55c3f6474a8f9adb482b6a8231 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:00.218229 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea180175-0d5b-4c08-ba7e-59b99873f6bf service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:00.229680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea180175-0d5b-4c08-ba7e-59b99873f6bf service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:00.330740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:00.331759 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:00.332191 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:00.332666 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] PATH_INFO: `/v3/users/6097b1efdce848b1b597b19be92ba3f2` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:00.335674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6097b1efdce848b1b597b19be92ba3f2)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:00.338662 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:00.579833 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6097b1efdce848b1b597b19be92ba3f2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.636056 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6097b1efdce848b1b597b19be92ba3f2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.662035 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6097b1efdce848b1b597b19be92ba3f2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.680525 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6097b1efdce848b1b597b19be92ba3f2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.681414 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-296bc90d-3334-4d24-854c-157429844901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6097b1efdce848b1b597b19be92ba3f2'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:00.694601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1393/2817] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:02:00 2022] DELETE /identity/v3/users/6097b1efdce848b1b597b19be92ba3f2 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:01.476492 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:01.476959 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:01.533366 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c64319-4ea3-414c-9390-fa1c4122e2bd admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:01.651421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:01.651862 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:01.652153 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:01.653026 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] PATH_INFO: `/v3/projects/a2a42db497fe4c3a999749d65ee98906` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:01.655034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a42db497fe4c3a999749d65ee98906)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:01.658006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:01.861661 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a42db497fe4c3a999749d65ee98906', 'request_id': 'req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:01.878678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a42db497fe4c3a999749d65ee98906', 'request_id': 'req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:01.961456 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a42db497fe4c3a999749d65ee98906', 'request_id': 'req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:01.976867 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] The token cache is being invalidate because project a2a42db497fe4c3a999749d65ee98906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:02:01.977311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:01.978009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-6b1c6364-2b16-4f75-bd82-3a3f32ba5c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:01.980424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1425/2818] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:02:01 2022] DELETE /identity/v3/projects/a2a42db497fe4c3a999749d65ee98906 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:01.992371 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de96a-50ce-4f15-af6d-d36081b45996 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:01.993084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de96a-50ce-4f15-af6d-d36081b45996 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:01.993084 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de96a-50ce-4f15-af6d-d36081b45996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:02.047944 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d09de96a-50ce-4f15-af6d-d36081b45996 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:02.049219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d09de96a-50ce-4f15-af6d-d36081b45996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:02.099708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:02.228320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1394/2819] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:02.255251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bb783-fb61-4152-b9d2-741900894a2e None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:02.256904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bb783-fb61-4152-b9d2-741900894a2e None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:02.256904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bb783-fb61-4152-b9d2-741900894a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:02.319582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-bf7bb783-fb61-4152-b9d2-741900894a2e None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:02.320997 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-bf7bb783-fb61-4152-b9d2-741900894a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:02.358992 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1426/2820] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:02.361994 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09de96a-50ce-4f15-af6d-d36081b45996 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:02.430889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09de96a-50ce-4f15-af6d-d36081b45996 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:02.430889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c88fc-2273-44de-a692-2c0a6ff46993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:02.430889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c88fc-2273-44de-a692-2c0a6ff46993 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:02.430889 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c88fc-2273-44de-a692-2c0a6ff46993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:02.431234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c88fc-2273-44de-a692-2c0a6ff46993 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:02.431234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c88fc-2273-44de-a692-2c0a6ff46993 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:02:02.431234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c88fc-2273-44de-a692-2c0a6ff46993 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:02.431234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c88fc-2273-44de-a692-2c0a6ff46993 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:02.431234 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1395/2821] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:02:02.434849 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bb783-fb61-4152-b9d2-741900894a2e None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:02.456901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bb783-fb61-4152-b9d2-741900894a2e None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:02.467803 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1cb0c-fdfc-4d5c-b190-97c50a6dbc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:02.468694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1cb0c-fdfc-4d5c-b190-97c50a6dbc06 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:02.468977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1cb0c-fdfc-4d5c-b190-97c50a6dbc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:02.469253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1cb0c-fdfc-4d5c-b190-97c50a6dbc06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:02.469939 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1cb0c-fdfc-4d5c-b190-97c50a6dbc06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:02.471931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1cb0c-fdfc-4d5c-b190-97c50a6dbc06 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:02.483002 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa1b85-70db-4d7a-9743-cb1ba182ad38 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:02.483527 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa1b85-70db-4d7a-9743-cb1ba182ad38 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:02.484106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa1b85-70db-4d7a-9743-cb1ba182ad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:02.540858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-d0fa1b85-70db-4d7a-9743-cb1ba182ad38 None None] MFA Rules not processed for user `34d452ce6a1245c192acf8c87651fe92`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:02.542113 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d0fa1b85-70db-4d7a-9743-cb1ba182ad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:02.635004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1396/2822] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:02:02.640155 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa1b85-70db-4d7a-9743-cb1ba182ad38 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:02.643316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d0fa1b85-70db-4d7a-9743-cb1ba182ad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:02.657592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa1b85-70db-4d7a-9743-cb1ba182ad38 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:02.671014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344adda4-1660-4f09-8035-2c6f1ab9eb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:02.671014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344adda4-1660-4f09-8035-2c6f1ab9eb69 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:02.671014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344adda4-1660-4f09-8035-2c6f1ab9eb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:02.671014 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344adda4-1660-4f09-8035-2c6f1ab9eb69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:02.671469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344adda4-1660-4f09-8035-2c6f1ab9eb69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:02.671469 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344adda4-1660-4f09-8035-2c6f1ab9eb69 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:02.754286 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1427/2823] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:02:02 2022] POST /identity/v3/projects => generated 373 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:02:02.757308 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1cb0c-fdfc-4d5c-b190-97c50a6dbc06 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:02.791261 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1397/2824] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:02:02 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:02:02.801018 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344adda4-1660-4f09-8035-2c6f1ab9eb69 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:02.805387 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:02.833538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1cb0c-fdfc-4d5c-b190-97c50a6dbc06 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:02.889119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344adda4-1660-4f09-8035-2c6f1ab9eb69 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:02.898071 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993c030-f19b-4421-a98a-a0d626d94ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:02.899232 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993c030-f19b-4421-a98a-a0d626d94ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:02.899752 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993c030-f19b-4421-a98a-a0d626d94ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:02.899993 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993c030-f19b-4421-a98a-a0d626d94ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:02.900689 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993c030-f19b-4421-a98a-a0d626d94ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:02.904463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993c030-f19b-4421-a98a-a0d626d94ba2 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:02.926965 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed747fb-7222-4fff-8088-19c99f809ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:02.927877 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed747fb-7222-4fff-8088-19c99f809ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:02.928095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed747fb-7222-4fff-8088-19c99f809ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:02.928451 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed747fb-7222-4fff-8088-19c99f809ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:02.929163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed747fb-7222-4fff-8088-19c99f809ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:02.931338 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed747fb-7222-4fff-8088-19c99f809ee3 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:03.036465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1398/2825] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:02:02 2022] POST /identity/v3/users => generated 296 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:02:03.043800 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993c030-f19b-4421-a98a-a0d626d94ba2 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:03.050153 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:03.052609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1428/2826] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:02:02 2022] POST /identity/v3/users => generated 287 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:02:03.056977 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993c030-f19b-4421-a98a-a0d626d94ba2 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:03.057191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed747fb-7222-4fff-8088-19c99f809ee3 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:03.062775 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d07db-70bb-4edc-b1ad-2788d2cc9749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:03.063708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d07db-70bb-4edc-b1ad-2788d2cc9749 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:03.064090 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d07db-70bb-4edc-b1ad-2788d2cc9749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:03.064468 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d07db-70bb-4edc-b1ad-2788d2cc9749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:03.065380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d07db-70bb-4edc-b1ad-2788d2cc9749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:03.073419 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d07db-70bb-4edc-b1ad-2788d2cc9749 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:03.073583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed747fb-7222-4fff-8088-19c99f809ee3 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:03.074750 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353d653-a70e-4767-83cd-cbb670c045a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:03.075369 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353d653-a70e-4767-83cd-cbb670c045a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:03.075872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353d653-a70e-4767-83cd-cbb670c045a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:03.076152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353d653-a70e-4767-83cd-cbb670c045a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:03.076954 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353d653-a70e-4767-83cd-cbb670c045a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:03.079155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353d653-a70e-4767-83cd-cbb670c045a8 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:03.082122 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1399/2827] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:02:03.088471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d07db-70bb-4edc-b1ad-2788d2cc9749 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:03.104107 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d07db-70bb-4edc-b1ad-2788d2cc9749 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:03.104219 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1429/2828] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:02:03.108133 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353d653-a70e-4767-83cd-cbb670c045a8 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:03.108616 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaeafe8-cfc2-4885-8b41-a5a71ac14b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:03.109515 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaeafe8-cfc2-4885-8b41-a5a71ac14b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:03.109891 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaeafe8-cfc2-4885-8b41-a5a71ac14b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:03.110240 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaeafe8-cfc2-4885-8b41-a5a71ac14b9c admin admin] PATH_INFO: `/v3/projects/5f356e7904cd4ca3b860f14ec9a6241e/users/85bb93d4f5534349b21c62f568ec0633/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:03.128511 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353d653-a70e-4767-83cd-cbb670c045a8 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:03.131428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaeafe8-cfc2-4885-8b41-a5a71ac14b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f356e7904cd4ca3b860f14ec9a6241e, user_id=85bb93d4f5534349b21c62f568ec0633, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:03.135198 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdd3f2-7048-4111-9643-15a645b30a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:03.137691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdd3f2-7048-4111-9643-15a645b30a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:03.137691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdd3f2-7048-4111-9643-15a645b30a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:03.137691 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdd3f2-7048-4111-9643-15a645b30a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:03.137968 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaeafe8-cfc2-4885-8b41-a5a71ac14b9c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:03.142115 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebdd3f2-7048-4111-9643-15a645b30a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:03.144269 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebdd3f2-7048-4111-9643-15a645b30a5f admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:03.169515 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1430/2829] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:02:03.173855 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdd3f2-7048-4111-9643-15a645b30a5f admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:03.186628 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdd3f2-7048-4111-9643-15a645b30a5f admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:03.193090 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c0d6b-713a-49d2-bfcc-3b915bebea7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:03.193852 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c0d6b-713a-49d2-bfcc-3b915bebea7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:03.194151 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c0d6b-713a-49d2-bfcc-3b915bebea7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:03.195131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c0d6b-713a-49d2-bfcc-3b915bebea7b admin admin] PATH_INFO: `/v3/projects/a4ab9457de96429abbe8fa5f6c8523b7/users/3ae1e5eb094b4d22a281504dd1698d26/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:03.216756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c0d6b-713a-49d2-bfcc-3b915bebea7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ab9457de96429abbe8fa5f6c8523b7, user_id=3ae1e5eb094b4d22a281504dd1698d26, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:03.219696 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c0d6b-713a-49d2-bfcc-3b915bebea7b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:03.234614 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1400/2830] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:02:03 2022] PUT /identity/v3/projects/5f356e7904cd4ca3b860f14ec9a6241e/users/85bb93d4f5534349b21c62f568ec0633/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:03.235601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaeafe8-cfc2-4885-8b41-a5a71ac14b9c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:03.332434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaeafe8-cfc2-4885-8b41-a5a71ac14b9c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:03.341782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f9d51-6de4-4c2f-9df5-253d69b07308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:03.342777 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f9d51-6de4-4c2f-9df5-253d69b07308 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:03.343238 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f9d51-6de4-4c2f-9df5-253d69b07308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:03.343677 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f9d51-6de4-4c2f-9df5-253d69b07308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:03.344494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f9d51-6de4-4c2f-9df5-253d69b07308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:03.344923 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1431/2831] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:02:03 2022] PUT /identity/v3/projects/a4ab9457de96429abbe8fa5f6c8523b7/users/3ae1e5eb094b4d22a281504dd1698d26/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:03.346475 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f9d51-6de4-4c2f-9df5-253d69b07308 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:03.349899 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bca8d4-a8ed-4276-93c2-f7849b27525d None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:03.350190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bca8d4-a8ed-4276-93c2-f7849b27525d None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:03.350466 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bca8d4-a8ed-4276-93c2-f7849b27525d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:03.360022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1401/2832] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:02:03.366178 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f9d51-6de4-4c2f-9df5-253d69b07308 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:03.379263 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f9d51-6de4-4c2f-9df5-253d69b07308 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:03.382000 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-c2bca8d4-a8ed-4276-93c2-f7849b27525d None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:03.383431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c2bca8d4-a8ed-4276-93c2-f7849b27525d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:03.459425 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1432/2833] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:03.470422 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bca8d4-a8ed-4276-93c2-f7849b27525d None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:03.472911 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-c2bca8d4-a8ed-4276-93c2-f7849b27525d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:03.487669 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e38570-344f-4cf0-8a31-3f31151b56a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:03.488464 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e38570-344f-4cf0-8a31-3f31151b56a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:03.488851 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e38570-344f-4cf0-8a31-3f31151b56a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:03.489233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e38570-344f-4cf0-8a31-3f31151b56a9 admin admin] PATH_INFO: `/v3/projects/5f356e7904cd4ca3b860f14ec9a6241e/users/85bb93d4f5534349b21c62f568ec0633/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:03.491369 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e38570-344f-4cf0-8a31-3f31151b56a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f356e7904cd4ca3b860f14ec9a6241e, user_id=85bb93d4f5534349b21c62f568ec0633, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:03.494281 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e38570-344f-4cf0-8a31-3f31151b56a9 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:03.512632 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1402/2834] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:02:03 2022] PUT /identity/v3/projects/5f356e7904cd4ca3b860f14ec9a6241e/users/85bb93d4f5534349b21c62f568ec0633/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:02:03.522134 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:03.548037 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bca8d4-a8ed-4276-93c2-f7849b27525d None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:03.615635 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37eff7-232d-4837-affd-a418094dc811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:03.616328 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37eff7-232d-4837-affd-a418094dc811 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:03.616704 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37eff7-232d-4837-affd-a418094dc811 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:03.617047 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37eff7-232d-4837-affd-a418094dc811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:03.710481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37eff7-232d-4837-affd-a418094dc811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:03.712540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37eff7-232d-4837-affd-a418094dc811 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:03.734400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1433/2835] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:02:05.555756 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74ca49-827a-4e94-be2a-850b649f8ff7 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:05.556526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74ca49-827a-4e94-be2a-850b649f8ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:05.556526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74ca49-827a-4e94-be2a-850b649f8ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:05.591250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-5d74ca49-827a-4e94-be2a-850b649f8ff7 None None] MFA Rules not processed for user `3ae1e5eb094b4d22a281504dd1698d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:05.591699 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5d74ca49-827a-4e94-be2a-850b649f8ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:05.675748 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:05.707325 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1403/2836] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:05.709044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37eff7-232d-4837-affd-a418094dc811 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:05.723065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37eff7-232d-4837-affd-a418094dc811 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:05.735214 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19feb96d-b2d4-4541-bad4-f9c57ba0460a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:05.735798 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feb96d-b2d4-4541-bad4-f9c57ba0460a admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:05.736082 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feb96d-b2d4-4541-bad4-f9c57ba0460a admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:05.736558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feb96d-b2d4-4541-bad4-f9c57ba0460a admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:05.737579 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19feb96d-b2d4-4541-bad4-f9c57ba0460a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:05.740549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19feb96d-b2d4-4541-bad4-f9c57ba0460a admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:05.885646 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1434/2837] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:02:05 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:02:05.890169 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74ca49-827a-4e94-be2a-850b649f8ff7 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:05.916490 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74ca49-827a-4e94-be2a-850b649f8ff7 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:06.040405 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c453408-3144-4458-bd12-309834de68cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:06.043674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c453408-3144-4458-bd12-309834de68cc admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:06.044462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c453408-3144-4458-bd12-309834de68cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:06.044462 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c453408-3144-4458-bd12-309834de68cc admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:06.046295 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c453408-3144-4458-bd12-309834de68cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:06.051665 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c453408-3144-4458-bd12-309834de68cc admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:06.259766 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1404/2838] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:02:05 2022] POST /identity/v3/users => generated 292 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:02:06.266480 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19feb96d-b2d4-4541-bad4-f9c57ba0460a admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:06.282576 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19feb96d-b2d4-4541-bad4-f9c57ba0460a admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:06.294739 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3904f77-2984-4f61-b733-fb9fe914783b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:06.296195 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3904f77-2984-4f61-b733-fb9fe914783b admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:06.296709 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3904f77-2984-4f61-b733-fb9fe914783b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:06.297320 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3904f77-2984-4f61-b733-fb9fe914783b admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:06.298163 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3904f77-2984-4f61-b733-fb9fe914783b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:06.300765 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3904f77-2984-4f61-b733-fb9fe914783b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:06.391986 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1435/2839] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:02:06.396145 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c453408-3144-4458-bd12-309834de68cc admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:06.424943 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c453408-3144-4458-bd12-309834de68cc admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:06.457733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b621c92-f12d-4e83-b2c8-de5952166af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:06.458562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621c92-f12d-4e83-b2c8-de5952166af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:06.458874 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621c92-f12d-4e83-b2c8-de5952166af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:06.459167 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621c92-f12d-4e83-b2c8-de5952166af8 admin admin] PATH_INFO: `/v3/projects/650f161f35cb42809adfbb72797a4394/users/cf8989b10874491e89ba9c2d8bcc4c57/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:06.539374 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b621c92-f12d-4e83-b2c8-de5952166af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=650f161f35cb42809adfbb72797a4394, user_id=cf8989b10874491e89ba9c2d8bcc4c57, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:06.543346 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b621c92-f12d-4e83-b2c8-de5952166af8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:06.640476 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1405/2840] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:02:06 2022] PUT /identity/v3/projects/650f161f35cb42809adfbb72797a4394/users/cf8989b10874491e89ba9c2d8bcc4c57/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:06.645623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3904f77-2984-4f61-b733-fb9fe914783b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:06.669307 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3904f77-2984-4f61-b733-fb9fe914783b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:06.743994 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a4171-3694-4851-8ff5-3405c4fe39e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:06.744653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a4171-3694-4851-8ff5-3405c4fe39e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:06.744790 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a4171-3694-4851-8ff5-3405c4fe39e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:06.745077 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a4171-3694-4851-8ff5-3405c4fe39e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:06.745740 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a4171-3694-4851-8ff5-3405c4fe39e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:06.748522 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a4171-3694-4851-8ff5-3405c4fe39e6 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:06.755236 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:06.760445 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1436/2841] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:02:06.764857 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b621c92-f12d-4e83-b2c8-de5952166af8 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:06.777114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b621c92-f12d-4e83-b2c8-de5952166af8 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:06.787274 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c8387-c469-4e2d-bf7b-cb87366c364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:06.789161 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c8387-c469-4e2d-bf7b-cb87366c364d admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:06.789453 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c8387-c469-4e2d-bf7b-cb87366c364d admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:06.789733 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c8387-c469-4e2d-bf7b-cb87366c364d admin admin] PATH_INFO: `/v3/projects/650f161f35cb42809adfbb72797a4394/users/cf8989b10874491e89ba9c2d8bcc4c57/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:06.792233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c8387-c469-4e2d-bf7b-cb87366c364d admin admin] RBAC: Authorizing `identity:create_grant(project_id=650f161f35cb42809adfbb72797a4394, user_id=cf8989b10874491e89ba9c2d8bcc4c57, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:06.797928 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c8387-c469-4e2d-bf7b-cb87366c364d admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:06.815218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1406/2842] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:02:06 2022] PUT /identity/v3/projects/650f161f35cb42809adfbb72797a4394/users/cf8989b10874491e89ba9c2d8bcc4c57/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:08.672538 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c77312-a83e-4437-9c8c-e816e0633f37 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:08.673034 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c77312-a83e-4437-9c8c-e816e0633f37 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:08.673290 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c77312-a83e-4437-9c8c-e816e0633f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:08.710475 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-28c77312-a83e-4437-9c8c-e816e0633f37 None None] MFA Rules not processed for user `cf8989b10874491e89ba9c2d8bcc4c57`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:08.712048 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-28c77312-a83e-4437-9c8c-e816e0633f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:08.794279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1437/2843] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:08.806324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c8387-c469-4e2d-bf7b-cb87366c364d admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:08.808241 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-cb7c8387-c469-4e2d-bf7b-cb87366c364d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:08.880492 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c8387-c469-4e2d-bf7b-cb87366c364d admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:08.896485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecb76f-f664-493d-8876-ed8874b6066a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:08.896485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecb76f-f664-493d-8876-ed8874b6066a service cinder] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:08.896485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecb76f-f664-493d-8876-ed8874b6066a service cinder] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:08.896485 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecb76f-f664-493d-8876-ed8874b6066a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:08.987323 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecb76f-f664-493d-8876-ed8874b6066a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:08.989406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:08.990128 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecb76f-f664-493d-8876-ed8874b6066a service cinder] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:09.009784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1407/2844] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:02:10.471901 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae78d-27bb-4bcb-a63f-5206fa348824 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:10.472455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae78d-27bb-4bcb-a63f-5206fa348824 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:10.472455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae78d-27bb-4bcb-a63f-5206fa348824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:10.500953 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-42bae78d-27bb-4bcb-a63f-5206fa348824 None None] MFA Rules not processed for user `b14c2eb2840b46538f169c39f200639f`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:10.502808 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-42bae78d-27bb-4bcb-a63f-5206fa348824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:10.579140 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1438/2845] 10.222.0.82 () {58 vars in 967 bytes} [Wed Aug 24 04:02:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:10.592684 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecb76f-f664-493d-8876-ed8874b6066a service cinder] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:10.596491 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-abecb76f-f664-493d-8876-ed8874b6066a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:10.717951 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecb76f-f664-493d-8876-ed8874b6066a service cinder] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:10.723751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6f439-5d21-4900-ba7e-557eb348c0ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad67f6ace29a4591bf2b8b4f119b3323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:10.728471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6f439-5d21-4900-ba7e-557eb348c0ad service swift] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:10.728471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6f439-5d21-4900-ba7e-557eb348c0ad service swift] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:10.728471 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6f439-5d21-4900-ba7e-557eb348c0ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:10.735547 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6f439-5d21-4900-ba7e-557eb348c0ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:10.737165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6f439-5d21-4900-ba7e-557eb348c0ad service swift] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:10.758771 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1408/2846] 10.222.0.82 () {60 vars in 1384 bytes} [Wed Aug 24 04:02:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 04:02:10.930670 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae78d-27bb-4bcb-a63f-5206fa348824 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:10.947167 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae78d-27bb-4bcb-a63f-5206fa348824 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:10.956955 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeaede-542d-4887-9d52-a7d5aa5a5df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:10.958268 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeaede-542d-4887-9d52-a7d5aa5a5df9 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:10.958391 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeaede-542d-4887-9d52-a7d5aa5a5df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:10.958701 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeaede-542d-4887-9d52-a7d5aa5a5df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:10.975758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfeaede-542d-4887-9d52-a7d5aa5a5df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:10.977725 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfeaede-542d-4887-9d52-a7d5aa5a5df9 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:10.991668 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1439/2847] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:02:11.514031 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b4250-53b2-4a82-aaba-cc218df5a8a0 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:11.514674 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b4250-53b2-4a82-aaba-cc218df5a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:11.515130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b4250-53b2-4a82-aaba-cc218df5a8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:11.562643 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c60b4250-53b2-4a82-aaba-cc218df5a8a0 None None] MFA Rules not processed for user `85bb93d4f5534349b21c62f568ec0633`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:11.565804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c60b4250-53b2-4a82-aaba-cc218df5a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:11.679840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1409/2848] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:11.696046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeaede-542d-4887-9d52-a7d5aa5a5df9 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:11.697545 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ccfeaede-542d-4887-9d52-a7d5aa5a5df9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:11.793228 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:11.813693 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeaede-542d-4887-9d52-a7d5aa5a5df9 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:11.820024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68b409-2cca-4a1a-b7fb-e424ccf36830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:11.820653 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68b409-2cca-4a1a-b7fb-e424ccf36830 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:11.820935 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68b409-2cca-4a1a-b7fb-e424ccf36830 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:11.821209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68b409-2cca-4a1a-b7fb-e424ccf36830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:11.882988 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68b409-2cca-4a1a-b7fb-e424ccf36830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:11.884604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68b409-2cca-4a1a-b7fb-e424ccf36830 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:11.894794 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1440/2849] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:02:14.167022 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b4250-53b2-4a82-aaba-cc218df5a8a0 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:14.169317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c60b4250-53b2-4a82-aaba-cc218df5a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:14.258614 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b4250-53b2-4a82-aaba-cc218df5a8a0 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:14.265185 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:14.266987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:14.266987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:14.266987 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] PATH_INFO: `/v3/users/8cfff1e957f1464dbee571606dc39624` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:14.267703 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cfff1e957f1464dbee571606dc39624)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:14.269663 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:14.285210 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:14.508840 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cfff1e957f1464dbee571606dc39624'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:14.594574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cfff1e957f1464dbee571606dc39624'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:14.608680 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cfff1e957f1464dbee571606dc39624'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:14.616159 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cfff1e957f1464dbee571606dc39624'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:14.616601 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cfff1e957f1464dbee571606dc39624'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:14.623647 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1410/2850] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:02:14 2022] DELETE /identity/v3/users/8cfff1e957f1464dbee571606dc39624 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:14.903341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68b409-2cca-4a1a-b7fb-e424ccf36830 service nova] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:14.916554 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68b409-2cca-4a1a-b7fb-e424ccf36830 service nova] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:14.988440 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:14.989091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:14.989379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:14.989662 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] PATH_INFO: `/v3/projects/3b064eb9c6944e11b79e161e4df5205a` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:14.990839 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b064eb9c6944e11b79e161e4df5205a)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:14.995700 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:15.125996 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b064eb9c6944e11b79e161e4df5205a', 'request_id': 'req-9a9d947a-10da-4944-bc15-e729fa0752e0'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:15.133025 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b064eb9c6944e11b79e161e4df5205a', 'request_id': 'req-9a9d947a-10da-4944-bc15-e729fa0752e0'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:15.233105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b064eb9c6944e11b79e161e4df5205a', 'request_id': 'req-9a9d947a-10da-4944-bc15-e729fa0752e0'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:15.254880 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] The token cache is being invalidate because project 3b064eb9c6944e11b79e161e4df5205a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:02:15.255288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:15.255951 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:15.257832 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1441/2851] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:02:14 2022] DELETE /identity/v3/projects/3b064eb9c6944e11b79e161e4df5205a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:24.655574 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=10,keystone:SELECT=22,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:25.250675 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:27.377606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:27.379573 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:27.396166 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c82a3-222d-4b10-9c90-26e0a3dd78e5 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:27.457722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:27.458739 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:27.459235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:27.459712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] PATH_INFO: `/v3/users/3ae1e5eb094b4d22a281504dd1698d26` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:27.461140 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ae1e5eb094b4d22a281504dd1698d26)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:27.463101 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:27.633039 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ae1e5eb094b4d22a281504dd1698d26'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:27.717909 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ae1e5eb094b4d22a281504dd1698d26'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:27.731628 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ae1e5eb094b4d22a281504dd1698d26'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:27.738722 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ae1e5eb094b4d22a281504dd1698d26'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:27.739470 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ae1e5eb094b4d22a281504dd1698d26'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:27.746212 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1411/2852] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:02:27 2022] DELETE /identity/v3/users/3ae1e5eb094b4d22a281504dd1698d26 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:28.002779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:28.004306 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:28.018902 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d947a-10da-4944-bc15-e729fa0752e0 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:28.085044 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:28.086105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:28.086385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:28.086688 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] PATH_INFO: `/v3/projects/a4ab9457de96429abbe8fa5f6c8523b7` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:28.087938 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ab9457de96429abbe8fa5f6c8523b7)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:28.090972 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:28.226462 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ab9457de96429abbe8fa5f6c8523b7', 'request_id': 'req-b9d790fb-9818-474d-b1f7-6b6a72f82957'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.232481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ab9457de96429abbe8fa5f6c8523b7', 'request_id': 'req-b9d790fb-9818-474d-b1f7-6b6a72f82957'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.311891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ab9457de96429abbe8fa5f6c8523b7', 'request_id': 'req-b9d790fb-9818-474d-b1f7-6b6a72f82957'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.323810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] The token cache is being invalidate because project a4ab9457de96429abbe8fa5f6c8523b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:02:28.324098 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.324760 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.326454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1442/2853] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:02:28 2022] DELETE /identity/v3/projects/a4ab9457de96429abbe8fa5f6c8523b7 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:28.330772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:28.332189 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:28.345025 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a08a9-e5da-45b6-999d-20480a15759a admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:28.405105 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:28.405822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:28.406045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:28.406320 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] PATH_INFO: `/v3/users/cf8989b10874491e89ba9c2d8bcc4c57` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:28.407494 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf8989b10874491e89ba9c2d8bcc4c57)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:28.409426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:28.433402 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:28.666418 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf8989b10874491e89ba9c2d8bcc4c57'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.748076 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf8989b10874491e89ba9c2d8bcc4c57'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.762428 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf8989b10874491e89ba9c2d8bcc4c57'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.769488 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf8989b10874491e89ba9c2d8bcc4c57'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.771440 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-33cd53d5-500c-438b-b72e-8e6e68caf945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf8989b10874491e89ba9c2d8bcc4c57'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:28.776096 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1412/2854] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:02:28 2022] DELETE /identity/v3/users/cf8989b10874491e89ba9c2d8bcc4c57 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:29.030604 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:29.032634 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:29.047045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d790fb-9818-474d-b1f7-6b6a72f82957 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:29.108531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:29.109174 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:29.109455 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:29.109730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] PATH_INFO: `/v3/projects/650f161f35cb42809adfbb72797a4394` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:29.110977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=650f161f35cb42809adfbb72797a4394)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:29.112904 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:29.226395 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:29.238813 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '650f161f35cb42809adfbb72797a4394', 'request_id': 'req-38aac003-85d0-4982-9ab0-aaafc3757e12'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:29.244261 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '650f161f35cb42809adfbb72797a4394', 'request_id': 'req-38aac003-85d0-4982-9ab0-aaafc3757e12'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:29.341186 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '650f161f35cb42809adfbb72797a4394', 'request_id': 'req-38aac003-85d0-4982-9ab0-aaafc3757e12'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:29.352374 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] The token cache is being invalidate because project 650f161f35cb42809adfbb72797a4394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:02:29.353386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:29.353386 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-38aac003-85d0-4982-9ab0-aaafc3757e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:02:29.355702 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1443/2855] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:02:29 2022] DELETE /identity/v3/projects/650f161f35cb42809adfbb72797a4394 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:29.361045 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936210a7-d8db-4abb-99b7-ad1c4324ce88 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:29.361347 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936210a7-d8db-4abb-99b7-ad1c4324ce88 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:29.361640 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936210a7-d8db-4abb-99b7-ad1c4324ce88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:29.382794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-936210a7-d8db-4abb-99b7-ad1c4324ce88 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:29.384028 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-936210a7-d8db-4abb-99b7-ad1c4324ce88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:29.444529 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1413/2856] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:29.455191 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083d753-c155-438a-a031-231543250be1 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:29.455474 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083d753-c155-438a-a031-231543250be1 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:29.455748 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083d753-c155-438a-a031-231543250be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:29.475651 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-9083d753-c155-438a-a031-231543250be1 None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:29.476869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9083d753-c155-438a-a031-231543250be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:29.485644 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1444/2857] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:29.492667 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936210a7-d8db-4abb-99b7-ad1c4324ce88 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:29.502895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936210a7-d8db-4abb-99b7-ad1c4324ce88 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:29.507465 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ca07a-6ed5-4303-9d3c-56aa38419497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:29.508061 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ca07a-6ed5-4303-9d3c-56aa38419497 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:29.508422 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ca07a-6ed5-4303-9d3c-56aa38419497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:29.508697 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ca07a-6ed5-4303-9d3c-56aa38419497 admin admin] PATH_INFO: `/v3/domains` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:29.509340 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ca07a-6ed5-4303-9d3c-56aa38419497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81943) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:02:29.509644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ca07a-6ed5-4303-9d3c-56aa38419497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:29.511885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ca07a-6ed5-4303-9d3c-56aa38419497 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:29.523901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1414/2858] 10.222.0.82 () {58 vars in 1153 bytes} [Wed Aug 24 04:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:02:29.539478 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083d753-c155-438a-a031-231543250be1 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:29.548816 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083d753-c155-438a-a031-231543250be1 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:29.553389 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb38ac-13f3-4b28-a495-f7b0fe498e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:29.553982 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb38ac-13f3-4b28-a495-f7b0fe498e1d admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:29.554257 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb38ac-13f3-4b28-a495-f7b0fe498e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:29.554558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb38ac-13f3-4b28-a495-f7b0fe498e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:29.555190 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb38ac-13f3-4b28-a495-f7b0fe498e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:29.557114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb38ac-13f3-4b28-a495-f7b0fe498e1d admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:29.754358 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1445/2859] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:02:29 2022] POST /identity/v3/projects => generated 373 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:02:29.761424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ca07a-6ed5-4303-9d3c-56aa38419497 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:29.767195 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ca07a-6ed5-4303-9d3c-56aa38419497 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:29.827526 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c7ef5-fcae-45c4-90eb-cbd5503363e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:29.828130 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c7ef5-fcae-45c4-90eb-cbd5503363e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:29.828413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c7ef5-fcae-45c4-90eb-cbd5503363e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:29.828694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c7ef5-fcae-45c4-90eb-cbd5503363e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:29.829331 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c7ef5-fcae-45c4-90eb-cbd5503363e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:29.831289 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c7ef5-fcae-45c4-90eb-cbd5503363e8 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:29.896608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1415/2860] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:02:29 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:02:29.900773 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb38ac-13f3-4b28-a495-f7b0fe498e1d admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:29.911680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb38ac-13f3-4b28-a495-f7b0fe498e1d admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:29.916442 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48809589-a6ee-458f-bc58-888c6b51b131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:29.917155 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48809589-a6ee-458f-bc58-888c6b51b131 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:29.917438 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48809589-a6ee-458f-bc58-888c6b51b131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:29.917845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48809589-a6ee-458f-bc58-888c6b51b131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:29.918531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48809589-a6ee-458f-bc58-888c6b51b131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:29.920124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48809589-a6ee-458f-bc58-888c6b51b131 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:29.930124 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1446/2861] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:02:29.934822 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c7ef5-fcae-45c4-90eb-cbd5503363e8 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:29.944858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c7ef5-fcae-45c4-90eb-cbd5503363e8 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:29.951420 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b265-3de5-4036-a4bf-356e903f32a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:29.952052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b265-3de5-4036-a4bf-356e903f32a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:29.952442 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b265-3de5-4036-a4bf-356e903f32a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:29.952865 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b265-3de5-4036-a4bf-356e903f32a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:29.953594 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b265-3de5-4036-a4bf-356e903f32a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:29.956231 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b265-3de5-4036-a4bf-356e903f32a0 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:29.968660 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1416/2862] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:02:29.972903 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48809589-a6ee-458f-bc58-888c6b51b131 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:29.983353 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48809589-a6ee-458f-bc58-888c6b51b131 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:29.987962 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52890afd-b26b-45e6-9bd2-869f1bbdfda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:29.988610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52890afd-b26b-45e6-9bd2-869f1bbdfda1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:29.988894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52890afd-b26b-45e6-9bd2-869f1bbdfda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:29.989170 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52890afd-b26b-45e6-9bd2-869f1bbdfda1 admin admin] PATH_INFO: `/v3/projects/c4943c05bd844f8d940def9908078748/users/aa39ed4cd12348c58eef09330af9c000/roles/5bf6686ae6624e84aba335589d8a98a4` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:30.005101 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52890afd-b26b-45e6-9bd2-869f1bbdfda1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4943c05bd844f8d940def9908078748, user_id=aa39ed4cd12348c58eef09330af9c000, role_id=5bf6686ae6624e84aba335589d8a98a4)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:30.007812 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52890afd-b26b-45e6-9bd2-869f1bbdfda1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:30.074881 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1447/2863] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:02:29 2022] PUT /identity/v3/projects/c4943c05bd844f8d940def9908078748/users/aa39ed4cd12348c58eef09330af9c000/roles/5bf6686ae6624e84aba335589d8a98a4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:30.080100 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801bdd4-36ac-43ab-a3fe-b176f96fcf8b None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:30.080424 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801bdd4-36ac-43ab-a3fe-b176f96fcf8b None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:30.080712 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801bdd4-36ac-43ab-a3fe-b176f96fcf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:30.089591 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:30.099862 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-5801bdd4-36ac-43ab-a3fe-b176f96fcf8b None None] MFA Rules not processed for user `c44d1cb246ea41a89355002d9d2481af`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:30.101121 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-5801bdd4-36ac-43ab-a3fe-b176f96fcf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:30.161052 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1417/2864] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:30.171281 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52890afd-b26b-45e6-9bd2-869f1bbdfda1 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:30.173005 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-52890afd-b26b-45e6-9bd2-869f1bbdfda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:30.229978 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52890afd-b26b-45e6-9bd2-869f1bbdfda1 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:30.233844 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf65303-d844-4bbb-8f18-43fa78186733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:30.234481 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf65303-d844-4bbb-8f18-43fa78186733 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:30.234797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf65303-d844-4bbb-8f18-43fa78186733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:30.235066 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf65303-d844-4bbb-8f18-43fa78186733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:30.244021 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf65303-d844-4bbb-8f18-43fa78186733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:30.246076 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf65303-d844-4bbb-8f18-43fa78186733 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:30.255632 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1448/2865] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:02:31.952770 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f871211-f4ce-4f31-a6f5-218aaca87851 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:31.953359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f871211-f4ce-4f31-a6f5-218aaca87851 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:31.953359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f871211-f4ce-4f31-a6f5-218aaca87851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:31.976732 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-4f871211-f4ce-4f31-a6f5-218aaca87851 None None] MFA Rules not processed for user `34d452ce6a1245c192acf8c87651fe92`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:31.978091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4f871211-f4ce-4f31-a6f5-218aaca87851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:32.049600 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1418/2866] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:32.101544 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf65303-d844-4bbb-8f18-43fa78186733 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:32.104830 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1bf65303-d844-4bbb-8f18-43fa78186733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:32.179698 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf65303-d844-4bbb-8f18-43fa78186733 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:32.187027 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89caed732274718aeaa0d8ffeef571b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:32.187655 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:32.187931 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:32.188199 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:32.189949 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:32.204744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:32.292046 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:32.296360 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:32.312240 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1449/2867] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:02:33.465620 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f871211-f4ce-4f31-a6f5-218aaca87851 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:33.467165 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4f871211-f4ce-4f31-a6f5-218aaca87851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:33.553547 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:33.555743 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:33.557783 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f871211-f4ce-4f31-a6f5-218aaca87851 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:33.566654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3763f-66e0-4dbb-82b0-8a542e5a70d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:33.568784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3763f-66e0-4dbb-82b0-8a542e5a70d0 service placement] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:33.568784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3763f-66e0-4dbb-82b0-8a542e5a70d0 service placement] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:33.568784 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3763f-66e0-4dbb-82b0-8a542e5a70d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:33.570549 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-84c3763f-66e0-4dbb-82b0-8a542e5a70d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:33.585955 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3763f-66e0-4dbb-82b0-8a542e5a70d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:33.587359 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452dc48-ca24-4a2b-9509-e4de780c2b19 service cinder] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:33.587864 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3763f-66e0-4dbb-82b0-8a542e5a70d0 service placement] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:33.601499 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1419/2868] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:02:33.602107 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7f9b7-d5db-4a58-a8ac-81b9072fdfb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:33.602956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7f9b7-d5db-4a58-a8ac-81b9072fdfb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:33.603401 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7f9b7-d5db-4a58-a8ac-81b9072fdfb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:33.603836 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7f9b7-d5db-4a58-a8ac-81b9072fdfb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:33.609350 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a6a7f9b7-d5db-4a58-a8ac-81b9072fdfb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:33.635045 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7f9b7-d5db-4a58-a8ac-81b9072fdfb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:33.637117 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7f9b7-d5db-4a58-a8ac-81b9072fdfb7 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:33.656209 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1450/2869] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:02:36.010816 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0503ec-317e-4ff4-b4b8-65cc50bdbf20 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:36.011588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0503ec-317e-4ff4-b4b8-65cc50bdbf20 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:36.011588 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0503ec-317e-4ff4-b4b8-65cc50bdbf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:36.053895 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-df0503ec-317e-4ff4-b4b8-65cc50bdbf20 None None] MFA Rules not processed for user `aa39ed4cd12348c58eef09330af9c000`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:36.055599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-df0503ec-317e-4ff4-b4b8-65cc50bdbf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:36.099175 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=100 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:36.144858 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1420/2870] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:36.149400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7f9b7-d5db-4a58-a8ac-81b9072fdfb7 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:36.163998 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7f9b7-d5db-4a58-a8ac-81b9072fdfb7 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:36.170792 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1c3f2-958c-4828-8fab-2ed5989ae99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:36.171699 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1c3f2-958c-4828-8fab-2ed5989ae99c admin admin] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:36.175030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1c3f2-958c-4828-8fab-2ed5989ae99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:36.175030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1c3f2-958c-4828-8fab-2ed5989ae99c admin admin] PATH_INFO: `/v3/projects` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:36.175030 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1c3f2-958c-4828-8fab-2ed5989ae99c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:36.175352 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1c3f2-958c-4828-8fab-2ed5989ae99c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:36.322262 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1451/2871] 10.222.0.82 () {60 vars in 1153 bytes} [Wed Aug 24 04:02:36 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:02:36.324413 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0503ec-317e-4ff4-b4b8-65cc50bdbf20 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:36.337260 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0503ec-317e-4ff4-b4b8-65cc50bdbf20 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:36.431524 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfba397-0de9-4eb7-aba2-ce78c0248c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:36.432216 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfba397-0de9-4eb7-aba2-ce78c0248c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:36.432576 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfba397-0de9-4eb7-aba2-ce78c0248c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:36.432882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfba397-0de9-4eb7-aba2-ce78c0248c62 admin admin] PATH_INFO: `/v3/users` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:36.433567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfba397-0de9-4eb7-aba2-ce78c0248c62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:36.435615 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfba397-0de9-4eb7-aba2-ce78c0248c62 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:36.565843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1421/2872] 10.222.0.82 () {60 vars in 1144 bytes} [Wed Aug 24 04:02:36 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:02:36.570006 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1c3f2-958c-4828-8fab-2ed5989ae99c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:36.586681 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1c3f2-958c-4828-8fab-2ed5989ae99c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:36.587694 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c637100-0160-4002-b372-749ed00b7d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:36.589020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c637100-0160-4002-b372-749ed00b7d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:36.589380 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c637100-0160-4002-b372-749ed00b7d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:36.589656 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c637100-0160-4002-b372-749ed00b7d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:36.590330 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c637100-0160-4002-b372-749ed00b7d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:36.591985 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c637100-0160-4002-b372-749ed00b7d19 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:36.603079 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1452/2873] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:02:36.607506 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfba397-0de9-4eb7-aba2-ce78c0248c62 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:36.617972 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfba397-0de9-4eb7-aba2-ce78c0248c62 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:36.626621 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e40d0-e7c5-480c-8886-f4ae9a4f5720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:36.627562 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e40d0-e7c5-480c-8886-f4ae9a4f5720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:36.627843 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e40d0-e7c5-480c-8886-f4ae9a4f5720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:36.628118 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e40d0-e7c5-480c-8886-f4ae9a4f5720 admin admin] PATH_INFO: `/v3/projects/1147aaee4d9a43a1923fa102c8e2cb6c/users/f16fe603465741a585a84d8b29e1ae95/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:36.644751 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e40d0-e7c5-480c-8886-f4ae9a4f5720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1147aaee4d9a43a1923fa102c8e2cb6c, user_id=f16fe603465741a585a84d8b29e1ae95, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:36.647590 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e40d0-e7c5-480c-8886-f4ae9a4f5720 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:36.790172 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1422/2874] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:02:36 2022] PUT /identity/v3/projects/1147aaee4d9a43a1923fa102c8e2cb6c/users/f16fe603465741a585a84d8b29e1ae95/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:36.791894 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c637100-0160-4002-b372-749ed00b7d19 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:36.802708 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c637100-0160-4002-b372-749ed00b7d19 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:36.874356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91edcb7-93f1-4f71-89c7-7932d3ae25c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:36.875017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91edcb7-93f1-4f71-89c7-7932d3ae25c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:36.875297 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91edcb7-93f1-4f71-89c7-7932d3ae25c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:36.875612 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91edcb7-93f1-4f71-89c7-7932d3ae25c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:36.876253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91edcb7-93f1-4f71-89c7-7932d3ae25c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:36.878288 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91edcb7-93f1-4f71-89c7-7932d3ae25c1 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:36.888642 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1453/2875] 10.222.0.82 () {58 vars in 1122 bytes} [Wed Aug 24 04:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:02:36.893043 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e40d0-e7c5-480c-8886-f4ae9a4f5720 admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:36.904659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e40d0-e7c5-480c-8886-f4ae9a4f5720 admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:36.909382 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f863139-c93e-4bfd-9587-3d73bdef16bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:36.910050 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f863139-c93e-4bfd-9587-3d73bdef16bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:36.910496 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f863139-c93e-4bfd-9587-3d73bdef16bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:36.910659 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f863139-c93e-4bfd-9587-3d73bdef16bf admin admin] PATH_INFO: `/v3/projects/1147aaee4d9a43a1923fa102c8e2cb6c/users/f16fe603465741a585a84d8b29e1ae95/roles/9e904216086e4a06abf9216b0c863f89` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:36.912625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f863139-c93e-4bfd-9587-3d73bdef16bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1147aaee4d9a43a1923fa102c8e2cb6c, user_id=f16fe603465741a585a84d8b29e1ae95, role_id=9e904216086e4a06abf9216b0c863f89)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:36.915412 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f863139-c93e-4bfd-9587-3d73bdef16bf admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:36.927705 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1423/2876] 10.222.0.82 () {60 vars in 1483 bytes} [Wed Aug 24 04:02:36 2022] PUT /identity/v3/projects/1147aaee4d9a43a1923fa102c8e2cb6c/users/f16fe603465741a585a84d8b29e1ae95/roles/9e904216086e4a06abf9216b0c863f89 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:02:37.220510 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91edcb7-93f1-4f71-89c7-7932d3ae25c1 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:37.302119 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91edcb7-93f1-4f71-89c7-7932d3ae25c1 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:37.305970 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:37.361355 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:37.362083 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:37.362429 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:37.362810 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:37.365216 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:37.386346 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:37.387925 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:37.409674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1454/2877] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:02:43.280404 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2004a-2f67-450b-9a3e-1c4232a8724f None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:43.280894 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2004a-2f67-450b-9a3e-1c4232a8724f None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:43.281114 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2004a-2f67-450b-9a3e-1c4232a8724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:43.327184 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-c4a2004a-2f67-450b-9a3e-1c4232a8724f None None] MFA Rules not processed for user `f16fe603465741a585a84d8b29e1ae95`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:43.328437 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-c4a2004a-2f67-450b-9a3e-1c4232a8724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:43.406275 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1424/2878] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:43.421250 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:43.422814 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:43.499758 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9975d-ab60-4e0c-8e65-3322dbcd90da service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:43.505797 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7310bdb-a3af-42b2-bb48-c451bdcc56d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:43.506529 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7310bdb-a3af-42b2-bb48-c451bdcc56d8 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:43.506926 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7310bdb-a3af-42b2-bb48-c451bdcc56d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:43.507278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7310bdb-a3af-42b2-bb48-c451bdcc56d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:43.566817 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7310bdb-a3af-42b2-bb48-c451bdcc56d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:43.568424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7310bdb-a3af-42b2-bb48-c451bdcc56d8 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:43.577741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1455/2879] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:02:46.456715 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f9505-a102-4321-b206-cd415ded7c50 None None] REQUEST_METHOD: `POST` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:46.457233 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f9505-a102-4321-b206-cd415ded7c50 None None] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:46.457343 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f9505-a102-4321-b206-cd415ded7c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:46.471509 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:46.479086 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.auth.core [None req-1e0f9505-a102-4321-b206-cd415ded7c50 None None] MFA Rules not processed for user `aa39ed4cd12348c58eef09330af9c000`. Rule list: `[]` (Enabled: `True`). {{(pid=81943) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:02:46.480423 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1e0f9505-a102-4321-b206-cd415ded7c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:46.487737 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1425/2880] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:02:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:02:53.243779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7310bdb-a3af-42b2-bb48-c451bdcc56d8 service nova] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:53.254278 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7310bdb-a3af-42b2-bb48-c451bdcc56d8 service nova] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:53.258094 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48abc7-e433-4387-ae77-8c2d0f080f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:53.258745 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48abc7-e433-4387-ae77-8c2d0f080f45 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:53.259033 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48abc7-e433-4387-ae77-8c2d0f080f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:53.259311 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48abc7-e433-4387-ae77-8c2d0f080f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:53.262920 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-6a48abc7-e433-4387-ae77-8c2d0f080f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:53.315891 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=100 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:02:53.334876 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48abc7-e433-4387-ae77-8c2d0f080f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:53.336334 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48abc7-e433-4387-ae77-8c2d0f080f45 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:53.347024 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1456/2881] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:02:55.390417 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f9505-a102-4321-b206-cd415ded7c50 None None] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:02:55.392023 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1e0f9505-a102-4321-b206-cd415ded7c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:55.408267 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f9505-a102-4321-b206-cd415ded7c50 None None] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:02:55.417202 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792a57c-aff1-455c-a134-4d5791e96339 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:02:55.417460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792a57c-aff1-455c-a134-4d5791e96339 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:02:55.417584 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792a57c-aff1-455c-a134-4d5791e96339 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:02:55.417845 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792a57c-aff1-455c-a134-4d5791e96339 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:02:55.419782 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-a792a57c-aff1-455c-a134-4d5791e96339 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:02:55.433007 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792a57c-aff1-455c-a134-4d5791e96339 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:02:55.434625 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792a57c-aff1-455c-a134-4d5791e96339 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:02:55.445460 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1426/2882] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:02:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:03:02.302027 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48abc7-e433-4387-ae77-8c2d0f080f45 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:03:02.311448 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48abc7-e433-4387-ae77-8c2d0f080f45 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:03:02.315095 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f45092-f71d-4dd2-bafd-089bc2d19d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:03:02.315697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f45092-f71d-4dd2-bafd-089bc2d19d60 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:03:02.315977 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f45092-f71d-4dd2-bafd-089bc2d19d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:03:02.316253 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f45092-f71d-4dd2-bafd-089bc2d19d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:03:02.317973 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-a5f45092-f71d-4dd2-bafd-089bc2d19d60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:03:02.328586 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f45092-f71d-4dd2-bafd-089bc2d19d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:03:02.330020 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f45092-f71d-4dd2-bafd-089bc2d19d60 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:03:02.338341 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1457/2883] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:03:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:03:05.440827 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=6 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:03:07.154613 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792a57c-aff1-455c-a134-4d5791e96339 service nova] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:03:07.166871 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792a57c-aff1-455c-a134-4d5791e96339 service nova] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:03:07.172825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544942ef-c915-441e-a8d9-a1014297ff10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:03:07.173605 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544942ef-c915-441e-a8d9-a1014297ff10 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:03:07.174047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544942ef-c915-441e-a8d9-a1014297ff10 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:03:07.174329 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544942ef-c915-441e-a8d9-a1014297ff10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:03:07.176945 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-544942ef-c915-441e-a8d9-a1014297ff10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:03:07.191280 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544942ef-c915-441e-a8d9-a1014297ff10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:03:07.192878 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544942ef-c915-441e-a8d9-a1014297ff10 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:03:07.204235 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1427/2884] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:03:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:03:12.334606 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=15 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:03:17.199256 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=6 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:03:18.508663 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f45092-f71d-4dd2-bafd-089bc2d19d60 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:03:18.522251 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f45092-f71d-4dd2-bafd-089bc2d19d60 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:03:18.528431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe623eb-6896-44d0-908d-0e22b07b1253 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:03:18.529050 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe623eb-6896-44d0-908d-0e22b07b1253 service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:03:18.529338 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe623eb-6896-44d0-908d-0e22b07b1253 service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:03:18.529622 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe623eb-6896-44d0-908d-0e22b07b1253 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:03:18.531441 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7fe623eb-6896-44d0-908d-0e22b07b1253 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:03:18.544963 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe623eb-6896-44d0-908d-0e22b07b1253 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:03:18.546574 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe623eb-6896-44d0-908d-0e22b07b1253 service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:03:18.558114 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1458/2885] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:03:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:03:21.172315 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544942ef-c915-441e-a8d9-a1014297ff10 service nova] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:03:21.181805 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544942ef-c915-441e-a8d9-a1014297ff10 service nova] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:03:21.185641 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77873da-a78b-4d66-96f3-fd8c7ceed2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:03:21.186252 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77873da-a78b-4d66-96f3-fd8c7ceed2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:03:21.186558 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77873da-a78b-4d66-96f3-fd8c7ceed2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:03:21.186842 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77873da-a78b-4d66-96f3-fd8c7ceed2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:03:21.196077 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77873da-a78b-4d66-96f3-fd8c7ceed2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:03:21.197518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77873da-a78b-4d66-96f3-fd8c7ceed2e6 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:03:21.206992 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1428/2886] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:03:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:03:28.552800 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=6 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:03:31.202804 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=6 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:04:57.641113 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0e6b5-8a77-49f0-b016-b0335af00a3b None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:04:57.641623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0e6b5-8a77-49f0-b016-b0335af00a3b None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:04:57.641730 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0e6b5-8a77-49f0-b016-b0335af00a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:04:57.660371 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-7cb0e6b5-8a77-49f0-b016-b0335af00a3b None None] MFA Rules not processed for user `34d452ce6a1245c192acf8c87651fe92`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:04:57.661627 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-7cb0e6b5-8a77-49f0-b016-b0335af00a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:04:57.716279 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1459/2887] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:04:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:05:07.709557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=25 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:05:30.953982 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77873da-a78b-4d66-96f3-fd8c7ceed2e6 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:05:30.964463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77873da-a78b-4d66-96f3-fd8c7ceed2e6 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:05:30.968610 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:05:30.969222 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:05:30.969567 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:05:30.970135 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:05:30.972583 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:05:30.985794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:05:30.985794 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:05:30.994599 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1429/2888] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:05:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:05:32.101983 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0e6b5-8a77-49f0-b016-b0335af00a3b None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:05:32.112494 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0e6b5-8a77-49f0-b016-b0335af00a3b None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:05:32.117017 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a556a-311f-47f3-807e-263d91190051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:05:32.117943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a556a-311f-47f3-807e-263d91190051 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:05:32.117943 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a556a-311f-47f3-807e-263d91190051 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:05:32.118162 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a556a-311f-47f3-807e-263d91190051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:05:32.120160 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-324a556a-311f-47f3-807e-263d91190051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:05:32.135958 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a556a-311f-47f3-807e-263d91190051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:05:32.137421 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a556a-311f-47f3-807e-263d91190051 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:05:32.149105 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1460/2889] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:05:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:05:40.991434 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=6 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:05:42.143731 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=6 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:06:17.339791 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:06:17.341342 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:06:17.411463 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25add4d9-18db-45f6-b329-4e9a8be00fae service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:06:17.416609 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34ece8-9bb1-40fe-b278-7985ddf51bf7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:06:17.417219 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34ece8-9bb1-40fe-b278-7985ddf51bf7 service placement] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:06:17.417502 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34ece8-9bb1-40fe-b278-7985ddf51bf7 service placement] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:06:17.418755 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34ece8-9bb1-40fe-b278-7985ddf51bf7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:06:17.420767 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-6b34ece8-9bb1-40fe-b278-7985ddf51bf7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:06:17.432700 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34ece8-9bb1-40fe-b278-7985ddf51bf7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:06:17.434882 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34ece8-9bb1-40fe-b278-7985ddf51bf7 service placement] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:06:17.445330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1430/2890] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:06:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:06:27.440888 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=32 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:06:58.565049 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20324a-2983-4693-b8df-9e483042ef08 None None] REQUEST_METHOD: `POST` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:06:58.565615 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20324a-2983-4693-b8df-9e483042ef08 None None] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:06:58.565756 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20324a-2983-4693-b8df-9e483042ef08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:06:58.584672 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.auth.core [None req-ca20324a-2983-4693-b8df-9e483042ef08 None None] MFA Rules not processed for user `34d452ce6a1245c192acf8c87651fe92`. Rule list: `[]` (Enabled: `True`). {{(pid=81942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:06:58.585907 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ca20324a-2983-4693-b8df-9e483042ef08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:06:58.592872 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1461/2891] 10.222.0.82 () {58 vars in 958 bytes} [Wed Aug 24 04:06:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:07:08.580102 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=3 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:07:10.271401 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34ece8-9bb1-40fe-b278-7985ddf51bf7 service placement] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:07:10.280952 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34ece8-9bb1-40fe-b278-7985ddf51bf7 service placement] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:07:10.285624 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59997ddb-f239-4f84-a134-a1f900d047bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:07:10.286254 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997ddb-f239-4f84-a134-a1f900d047bc service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:07:10.286569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997ddb-f239-4f84-a134-a1f900d047bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:07:10.286844 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997ddb-f239-4f84-a134-a1f900d047bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:07:10.288750 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-59997ddb-f239-4f84-a134-a1f900d047bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:07:10.300859 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59997ddb-f239-4f84-a134-a1f900d047bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:07:10.302243 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59997ddb-f239-4f84-a134-a1f900d047bc service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:07:10.311518 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1431/2892] 10.222.0.82 () {60 vars in 1396 bytes} [Wed Aug 24 04:07:10 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:07:16.023424 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20324a-2983-4693-b8df-9e483042ef08 None None] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:07:16.036362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20324a-2983-4693-b8df-9e483042ef08 None None] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:07:16.041525 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f564e80-be88-40e3-ab53-cf3095489682 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:07:16.042239 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f564e80-be88-40e3-ab53-cf3095489682 service placement] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:07:16.042549 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f564e80-be88-40e3-ab53-cf3095489682 service placement] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:07:16.042834 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f564e80-be88-40e3-ab53-cf3095489682 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:07:16.044590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-9f564e80-be88-40e3-ab53-cf3095489682 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:07:16.058706 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f564e80-be88-40e3-ab53-cf3095489682 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:07:16.060324 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f564e80-be88-40e3-ab53-cf3095489682 service placement] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:07:16.071682 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1462/2893] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:07:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:07:20.066330 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59997ddb-f239-4f84-a134-a1f900d047bc service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:07:20.084055 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59997ddb-f239-4f84-a134-a1f900d047bc service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:07:20.090324 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158c03b-fa23-4b33-a37d-2ee5f169662f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:07:20.091215 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158c03b-fa23-4b33-a37d-2ee5f169662f service placement] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:07:20.091694 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158c03b-fa23-4b33-a37d-2ee5f169662f service placement] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:07:20.092137 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158c03b-fa23-4b33-a37d-2ee5f169662f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:07:20.094163 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-1158c03b-fa23-4b33-a37d-2ee5f169662f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:07:20.109885 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158c03b-fa23-4b33-a37d-2ee5f169662f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:07:20.113359 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158c03b-fa23-4b33-a37d-2ee5f169662f service placement] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:07:20.136000 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1432/2894] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:07:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:07:20.215091 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f564e80-be88-40e3-ab53-cf3095489682 service placement] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:07:20.298054 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f564e80-be88-40e3-ab53-cf3095489682 service placement] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:07:20.303981 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:07:20.310078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:07:20.310078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:07:20.310078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] PATH_INFO: `/v3/users/aa39ed4cd12348c58eef09330af9c000` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:07:20.310078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa39ed4cd12348c58eef09330af9c000)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:07:20.310078 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:07:20.519323 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa39ed4cd12348c58eef09330af9c000'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:20.520159 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa39ed4cd12348c58eef09330af9c000'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:20.587482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa39ed4cd12348c58eef09330af9c000'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:20.595806 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa39ed4cd12348c58eef09330af9c000'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:20.614372 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa39ed4cd12348c58eef09330af9c000'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:20.623835 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1463/2895] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:07:20 2022] DELETE /identity/v3/users/aa39ed4cd12348c58eef09330af9c000 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:07:20.935592 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158c03b-fa23-4b33-a37d-2ee5f169662f service placement] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:07:20.945704 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158c03b-fa23-4b33-a37d-2ee5f169662f service placement] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:07:21.044457 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:07:21.045126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:07:21.045398 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:07:21.045931 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] PATH_INFO: `/v3/projects/c4943c05bd844f8d940def9908078748` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:07:21.047126 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4943c05bd844f8d940def9908078748)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:07:21.049119 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:07:21.200973 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4943c05bd844f8d940def9908078748', 'request_id': 'req-25f7f80d-7998-4418-bd81-97a1c8ddc81b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.278786 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4943c05bd844f8d940def9908078748', 'request_id': 'req-25f7f80d-7998-4418-bd81-97a1c8ddc81b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.286443 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4943c05bd844f8d940def9908078748', 'request_id': 'req-25f7f80d-7998-4418-bd81-97a1c8ddc81b'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.298298 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] The token cache is being invalidate because project c4943c05bd844f8d940def9908078748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:07:21.298773 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.300826 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.302937 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1433/2896] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:07:20 2022] DELETE /identity/v3/projects/c4943c05bd844f8d940def9908078748 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:07:21.307400 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:07:21.310697 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:07:21.324154 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592029-37c4-4c80-8410-dfdabd60ab0c admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:07:21.392513 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:07:21.393582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:07:21.393582 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:07:21.393845 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] PATH_INFO: `/v3/users/f16fe603465741a585a84d8b29e1ae95` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:07:21.395332 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16fe603465741a585a84d8b29e1ae95)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:07:21.397322 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:07:21.405397 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:07:21.598072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16fe603465741a585a84d8b29e1ae95'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.598784 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16fe603465741a585a84d8b29e1ae95'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.672509 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16fe603465741a585a84d8b29e1ae95'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.681356 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16fe603465741a585a84d8b29e1ae95'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.694256 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16fe603465741a585a84d8b29e1ae95'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:21.700379 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1464/2897] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:07:21 2022] DELETE /identity/v3/users/f16fe603465741a585a84d8b29e1ae95 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:07:21.942614 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:07:21.944091 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:07:21.960370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f80d-7998-4418-bd81-97a1c8ddc81b admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:07:22.020528 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:07:22.021332 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:07:22.021474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:07:22.021746 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] PATH_INFO: `/v3/projects/1147aaee4d9a43a1923fa102c8e2cb6c` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:07:22.022930 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1147aaee4d9a43a1923fa102c8e2cb6c)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:07:22.024887 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:07:22.031608 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:07:22.153708 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1147aaee4d9a43a1923fa102c8e2cb6c', 'request_id': 'req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:22.230316 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1147aaee4d9a43a1923fa102c8e2cb6c', 'request_id': 'req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:22.237901 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1147aaee4d9a43a1923fa102c8e2cb6c', 'request_id': 'req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:22.248276 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] The token cache is being invalidate because project 1147aaee4d9a43a1923fa102c8e2cb6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:07:22.248569 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:22.252497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:07:22.252497 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1434/2898] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:07:21 2022] DELETE /identity/v3/projects/1147aaee4d9a43a1923fa102c8e2cb6c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:07:31.696741 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:07:32.245230 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:08:20.998912 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:08:21.002941 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:08:21.081823 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa6943-bf4b-4c0b-8937-2b53458ec13b admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:08:21.132678 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:08:21.133339 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:08:21.133657 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:08:21.133956 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:08:21.135779 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:08:21.145488 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:08:21.147007 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:08:21.155394 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1465/2899] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:08:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:08:31.151557 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=46 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:08:49.216334 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:08:49.218923 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:08:49.304187 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923ca05-5c2d-45b4-b5b6-a79218fc75ce admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:08:49.310644 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:08:49.311370 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:08:49.311656 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:08:49.311932 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:08:49.313678 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:08:49.327772 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:08:49.329426 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:08:49.341394 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1435/2900] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:08:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:08:49.685905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:08:49.687531 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:08:49.698846 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da6f9c-3072-4f58-a054-4c4eec45d471 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:08:49.702482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9615e-c437-4ead-97a3-0e5b60705dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:08:49.703156 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9615e-c437-4ead-97a3-0e5b60705dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:08:49.703446 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9615e-c437-4ead-97a3-0e5b60705dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:08:49.703715 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9615e-c437-4ead-97a3-0e5b60705dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:08:49.705624 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-82e9615e-c437-4ead-97a3-0e5b60705dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:08:49.721277 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9615e-c437-4ead-97a3-0e5b60705dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:08:49.722680 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9615e-c437-4ead-97a3-0e5b60705dd4 service neutron] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:08:49.735752 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1466/2901] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:08:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:08:50.942164 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:08:50.943636 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:08:50.955498 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee81411-ba90-4c16-a4f1-6eca3ddb76c4 service nova] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:08:50.959213 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4685e20-53d5-435d-8e69-b61e5136d783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:08:50.959821 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4685e20-53d5-435d-8e69-b61e5136d783 service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:08:50.960106 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4685e20-53d5-435d-8e69-b61e5136d783 service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:08:50.960380 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4685e20-53d5-435d-8e69-b61e5136d783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:08:50.962173 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-d4685e20-53d5-435d-8e69-b61e5136d783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:08:50.977244 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4685e20-53d5-435d-8e69-b61e5136d783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:08:50.979313 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4685e20-53d5-435d-8e69-b61e5136d783 service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:08:50.990702 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1436/2902] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:08:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:08:51.824484 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9615e-c437-4ead-97a3-0e5b60705dd4 service neutron] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:08:51.836340 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9615e-c437-4ead-97a3-0e5b60705dd4 service neutron] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:08:51.842376 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f34fe3fe912498b90dbf950e7fc999d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:08:51.843014 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:08:51.843294 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:08:51.843575 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:08:51.845284 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:08:51.858540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:08:51.860127 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:08:51.871610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1467/2903] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:08:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:09:00.985582 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=40 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:09:01.866558 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=12 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:09:02.715348 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4685e20-53d5-435d-8e69-b61e5136d783 service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:09:02.724627 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4685e20-53d5-435d-8e69-b61e5136d783 service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:09:02.728368 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e2649-6104-4a48-98e7-5e45ffd4272a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81cc77e85f84d358471dd9efab45cd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:09:02.728965 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e2649-6104-4a48-98e7-5e45ffd4272a service neutron] REQUEST_METHOD: `GET` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:09:02.729250 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e2649-6104-4a48-98e7-5e45ffd4272a service neutron] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:09:02.729587 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e2649-6104-4a48-98e7-5e45ffd4272a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:09:02.731377 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-fb0e2649-6104-4a48-98e7-5e45ffd4272a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:09:02.797218 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e2649-6104-4a48-98e7-5e45ffd4272a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:09:02.798707 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e2649-6104-4a48-98e7-5e45ffd4272a service neutron] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:09:02.809004 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1437/2904] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:09:10.271918 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:09:10.274012 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:09:10.294207 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf4ecf-1de1-4ce0-a6ff-e6ab458e814e service nova] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:09:10.299208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:09:10.299435 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:09:10.299720 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:09:10.300001 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] PATH_INFO: `/v3/users/34d452ce6a1245c192acf8c87651fe92` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:09:10.301208 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d452ce6a1245c192acf8c87651fe92)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:09:10.303224 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:09:10.535674 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d452ce6a1245c192acf8c87651fe92'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:10.536431 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d452ce6a1245c192acf8c87651fe92'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:10.618623 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d452ce6a1245c192acf8c87651fe92'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:10.626344 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d452ce6a1245c192acf8c87651fe92'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:10.639609 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d452ce6a1245c192acf8c87651fe92'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:10.645636 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1468/2905] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:09:10 2022] DELETE /identity/v3/users/34d452ce6a1245c192acf8c87651fe92 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:09:10.920092 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e2649-6104-4a48-98e7-5e45ffd4272a service neutron] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:09:10.931326 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e2649-6104-4a48-98e7-5e45ffd4272a service neutron] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:09:11.004317 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:09:11.004974 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:09:11.005262 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:09:11.005542 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] PATH_INFO: `/v3/projects/8d390e7ad1454a8182258cd72bea4fa5` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:09:11.006740 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d390e7ad1454a8182258cd72bea4fa5)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:09:11.009152 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:09:11.158319 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d390e7ad1454a8182258cd72bea4fa5', 'request_id': 'req-b5de211f-7982-4051-ae2d-da76cc65ed0c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.241239 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d390e7ad1454a8182258cd72bea4fa5', 'request_id': 'req-b5de211f-7982-4051-ae2d-da76cc65ed0c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.248143 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d390e7ad1454a8182258cd72bea4fa5', 'request_id': 'req-b5de211f-7982-4051-ae2d-da76cc65ed0c'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.258753 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] The token cache is being invalidate because project 8d390e7ad1454a8182258cd72bea4fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:09:11.259047 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.259823 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.261416 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1438/2906] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:09:10 2022] DELETE /identity/v3/projects/8d390e7ad1454a8182258cd72bea4fa5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:09:11.265454 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:09:11.266900 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:09:11.281072 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d4caf-56e4-4627-a796-2f9e9a2d4daa admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:09:11.342273 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:09:11.343186 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:09:11.343468 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:09:11.343744 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] PATH_INFO: `/v3/users/85bb93d4f5534349b21c62f568ec0633` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:09:11.344916 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85bb93d4f5534349b21c62f568ec0633)` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:09:11.346869 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:09:11.559482 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85bb93d4f5534349b21c62f568ec0633'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.560152 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85bb93d4f5534349b21c62f568ec0633'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.654362 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85bb93d4f5534349b21c62f568ec0633'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.662590 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85bb93d4f5534349b21c62f568ec0633'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.670974 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:09:11.676063 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.notifications [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85bb93d4f5534349b21c62f568ec0633'} {{(pid=81942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:11.682065 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1469/2907] 10.222.0.82 () {58 vars in 1224 bytes} [Wed Aug 24 04:09:11 2022] DELETE /identity/v3/users/85bb93d4f5534349b21c62f568ec0633 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:09:11.965237 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] Authenticating user token {{(pid=81943) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:09:11.966711 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.fernet_utils [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81943) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:09:11.980877 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de211f-7982-4051-ae2d-da76cc65ed0c admin admin] Validating token access rules against request {{(pid=81943) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:09:12.003631 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:09:12.057825 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c44d1cb246ea41a89355002d9d2481af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0dfb110c668f41649b36f4c725749a3a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81943) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:09:12.058474 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:09:12.058799 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:09:12.059078 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] PATH_INFO: `/v3/projects/5f356e7904cd4ca3b860f14ec9a6241e` {{(pid=81943) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:09:12.060224 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f356e7904cd4ca3b860f14ec9a6241e)` {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:09:12.062150 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] RBAC: Authorization granted {{(pid=81943) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:09:12.198269 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f356e7904cd4ca3b860f14ec9a6241e', 'request_id': 'req-c536bc9e-14ac-414e-a3d3-3556d1ab3418'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:12.361406 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f356e7904cd4ca3b860f14ec9a6241e', 'request_id': 'req-c536bc9e-14ac-414e-a3d3-3556d1ab3418'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:12.369654 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f356e7904cd4ca3b860f14ec9a6241e', 'request_id': 'req-c536bc9e-14ac-414e-a3d3-3556d1ab3418'} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:12.380314 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] The token cache is being invalidate because project 5f356e7904cd4ca3b860f14ec9a6241e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81943) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:09:12.380606 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:12.381258 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG keystone.notifications [None req-c536bc9e-14ac-414e-a3d3-3556d1ab3418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81943) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:09:12.383134 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: [pid: 81943|app: 0|req: 1439/2908] 10.222.0.82 () {58 vars in 1233 bytes} [Wed Aug 24 04:09:11 2022] DELETE /identity/v3/projects/5f356e7904cd4ca3b860f14ec9a6241e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:09:21.825583 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:09:22.377158 dv-cinder-854260-1-iscsi devstack@keystone.service[81943]: DEBUG dbcounter [-] [81943] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:11:22.295385 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] Authenticating user token {{(pid=81942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:11:22.296883 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:11:22.439129 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e0dd9-e199-454e-8fcc-0b4a799b1912 admin admin] Validating token access rules against request {{(pid=81942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:11:22.497905 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449da0c-4f89-456c-9645-ba4dbfcbb675 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cff23ae056ff452588cd0b6520e96962', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a94ab421b414abb89ad09b813fce957', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:11:22.498540 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449da0c-4f89-456c-9645-ba4dbfcbb675 service placement] REQUEST_METHOD: `GET` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:11:22.498825 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449da0c-4f89-456c-9645-ba4dbfcbb675 service placement] SCRIPT_NAME: `/identity` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:11:22.499131 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449da0c-4f89-456c-9645-ba4dbfcbb675 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:11:22.500828 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.fernet_utils [None req-1449da0c-4f89-456c-9645-ba4dbfcbb675 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:11:22.513009 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449da0c-4f89-456c-9645-ba4dbfcbb675 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:11:22.514610 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449da0c-4f89-456c-9645-ba4dbfcbb675 service placement] RBAC: Authorization granted {{(pid=81942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:11:22.524520 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: [pid: 81942|app: 0|req: 1470/2909] 10.222.0.82 () {60 vars in 1365 bytes} [Wed Aug 24 04:11:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:11:32.520289 dv-cinder-854260-1-iscsi devstack@keystone.service[81942]: DEBUG dbcounter [-] [81942] Writing DB stats keystone:SELECT=89 {{(pid=81942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}